Sep 17 20:48:54.248700 ip-10-0-132-201 systemd[1]: Starting Kubernetes Kubelet...
Sep 17 20:48:54.668243 ip-10-0-132-201 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 20:48:54.668243 ip-10-0-132-201 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 17 20:48:54.668243 ip-10-0-132-201 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 20:48:54.668243 ip-10-0-132-201 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 17 20:48:54.668243 ip-10-0-132-201 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 20:48:54.669294 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.669208    2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 17 20:48:54.673459 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673443    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 20:48:54.673459 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673459    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 20:48:54.673523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673463    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 20:48:54.673523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673466    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 20:48:54.673523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673471    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 20:48:54.673523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673475    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 20:48:54.673523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673478    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 20:48:54.673523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673481    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 20:48:54.673523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673484    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 20:48:54.673523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673487    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 20:48:54.673523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673490    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 20:48:54.673523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673492    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 20:48:54.673523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673495    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 20:48:54.673523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673498    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 20:48:54.673523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673507    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 20:48:54.673523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673510    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 20:48:54.673523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673513    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 20:48:54.673523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673515    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 20:48:54.673523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673518    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 20:48:54.673523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673520    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 20:48:54.673523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673525    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 20:48:54.673968 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673529    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 20:48:54.673968 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673532    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 20:48:54.673968 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673536    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 20:48:54.673968 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673539    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 20:48:54.673968 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673542    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 20:48:54.673968 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673545    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 20:48:54.673968 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673547    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 20:48:54.673968 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673551    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 20:48:54.673968 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673555    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 20:48:54.673968 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673558    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 20:48:54.673968 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673561    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 20:48:54.673968 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673564    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 20:48:54.673968 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673567    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 20:48:54.673968 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673570    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 20:48:54.673968 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673573    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 20:48:54.673968 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673576    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 20:48:54.673968 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673579    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 20:48:54.673968 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673582    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 20:48:54.673968 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673585    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 20:48:54.673968 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673588    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 20:48:54.674494 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673590    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 20:48:54.674494 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673593    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 20:48:54.674494 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673595    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 20:48:54.674494 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673598    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 20:48:54.674494 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673600    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 20:48:54.674494 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673603    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 20:48:54.674494 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673605    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 20:48:54.674494 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673607    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 20:48:54.674494 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673610    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 20:48:54.674494 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673612    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 20:48:54.674494 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673615    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 20:48:54.674494 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673617    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 20:48:54.674494 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673621    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 20:48:54.674494 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673624    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 20:48:54.674494 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673627    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 20:48:54.674494 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673630    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 20:48:54.674494 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673632    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 20:48:54.674494 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673635    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 20:48:54.674494 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673637    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 20:48:54.674494 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673640    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 20:48:54.674972 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673644    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 20:48:54.674972 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673648    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 20:48:54.674972 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673652    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 20:48:54.674972 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673655    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 20:48:54.674972 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673657    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 20:48:54.674972 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673660    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 20:48:54.674972 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673662    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 20:48:54.674972 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673665    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 20:48:54.674972 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673667    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 20:48:54.674972 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673669    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 20:48:54.674972 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673673    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 20:48:54.674972 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673676    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 20:48:54.674972 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673679    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 20:48:54.674972 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673681    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 20:48:54.674972 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673684    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 20:48:54.674972 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673686    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 20:48:54.674972 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673689    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 20:48:54.674972 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673691    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 20:48:54.674972 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673694    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 20:48:54.675452 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673696    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 20:48:54.675452 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673699    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 20:48:54.675452 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673702    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 20:48:54.675452 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673704    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 20:48:54.675452 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673707    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 20:48:54.675452 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673710    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 20:48:54.675452 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673716    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 20:48:54.675452 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673719    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 20:48:54.675452 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673721    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 20:48:54.675452 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673724    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 20:48:54.675452 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673727    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 20:48:54.675452 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673730    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 20:48:54.675452 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673733    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 20:48:54.675452 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673735    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 20:48:54.675452 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673738    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 20:48:54.675452 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673740    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 20:48:54.675452 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673743    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 20:48:54.675452 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673745    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 20:48:54.675452 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.673748    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 20:48:54.675452 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675398    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 20:48:54.675929 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675406    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 20:48:54.675929 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675410    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 20:48:54.675929 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675413    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 20:48:54.675929 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675416    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 20:48:54.675929 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675420    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 20:48:54.675929 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675423    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 20:48:54.675929 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675425    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 20:48:54.675929 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675428    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 20:48:54.675929 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675431    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 20:48:54.675929 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675433    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 20:48:54.675929 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675436    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 20:48:54.675929 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675439    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 20:48:54.675929 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675443    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 20:48:54.675929 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675447    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 20:48:54.675929 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675450    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 20:48:54.675929 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675453    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 20:48:54.675929 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675456    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 20:48:54.675929 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675459    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 20:48:54.675929 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675462    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 20:48:54.676416 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675466    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 20:48:54.676416 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675469    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 20:48:54.676416 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675472    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 20:48:54.676416 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675474    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 20:48:54.676416 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675477    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 20:48:54.676416 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675480    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 20:48:54.676416 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675482    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 20:48:54.676416 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675484    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 20:48:54.676416 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675487    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 20:48:54.676416 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675489    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 20:48:54.676416 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675492    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 20:48:54.676416 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675494    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 20:48:54.676416 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675497    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 20:48:54.676416 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675499    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 20:48:54.676416 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675504    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 20:48:54.676416 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675507    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 20:48:54.676416 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675510    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 20:48:54.676416 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675512    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 20:48:54.676416 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675519    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 20:48:54.676416 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675522    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 20:48:54.676901 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675525    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 20:48:54.676901 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675528    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 20:48:54.676901 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675530    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 20:48:54.676901 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675533    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 20:48:54.676901 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675536    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 20:48:54.676901 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675539    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 20:48:54.676901 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675541    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 20:48:54.676901 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675544    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 20:48:54.676901 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675547    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 20:48:54.676901 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675549    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 20:48:54.676901 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675552    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 20:48:54.676901 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675555    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 20:48:54.676901 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675558    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 20:48:54.676901 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675560    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 20:48:54.676901 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675563    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 20:48:54.676901 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675566    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 20:48:54.676901 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675570    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 20:48:54.676901 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675574    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 20:48:54.676901 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675577    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 20:48:54.677362 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675579    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 20:48:54.677362 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675583    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 20:48:54.677362 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675587    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 20:48:54.677362 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675590    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 20:48:54.677362 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675593    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 20:48:54.677362 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675596    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 20:48:54.677362 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675600    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 20:48:54.677362 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675604    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 20:48:54.677362 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675608    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 20:48:54.677362 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675611    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 20:48:54.677362 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675614    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 20:48:54.677362 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675616    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 20:48:54.677362 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675625    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 20:48:54.677362 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675628    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 20:48:54.677362 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675631    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 20:48:54.677362 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675634    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 20:48:54.677362 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675636    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 20:48:54.677362 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675639    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 20:48:54.677362 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675642    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 20:48:54.677362 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675645    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 20:48:54.677887 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675648    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 20:48:54.677887 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675651    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 20:48:54.677887 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675653    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 20:48:54.677887 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675656    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 20:48:54.677887 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675659    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 20:48:54.677887 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675661    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 20:48:54.677887 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675664    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 20:48:54.677887 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675666    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 20:48:54.677887 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675669    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 20:48:54.677887 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675672    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 20:48:54.677887 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675674    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 20:48:54.677887 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675677    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 20:48:54.677887 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675679    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 20:48:54.677887 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675682    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 20:48:54.677887 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675684    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 20:48:54.677887 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675687    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 20:48:54.677887 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675690    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 20:48:54.677887 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675692    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 20:48:54.677887 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675695    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 20:48:54.677887 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.675699    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 20:48:54.678402 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676079    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 20:48:54.678402 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676084    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 20:48:54.678402 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676087    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 20:48:54.678402 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676090    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 20:48:54.678402 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676093    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 20:48:54.678402 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676096    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 20:48:54.678402 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676099    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 20:48:54.678402 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676101    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 20:48:54.678402 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676104    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 20:48:54.678402 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676107    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 20:48:54.678402 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676109    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 20:48:54.678402 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676112    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 20:48:54.678402 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676114    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 20:48:54.678402 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676118    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 20:48:54.678402 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676122    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 20:48:54.678402 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676125    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 20:48:54.678402 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676128    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 20:48:54.678402 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676131    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 20:48:54.678402 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676134    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 20:48:54.678402 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676136    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 20:48:54.678895 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676139    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 20:48:54.678895 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676142    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 20:48:54.678895 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676144    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 20:48:54.678895 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676147    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 20:48:54.678895 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676149    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 20:48:54.678895 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676152    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 20:48:54.678895 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676155    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 20:48:54.678895 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676157    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 20:48:54.678895 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676160    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 20:48:54.678895 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676162    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 20:48:54.678895 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676166    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 20:48:54.678895 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676170    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 20:48:54.678895 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676173    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 20:48:54.678895 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676176    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 20:48:54.678895 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676178    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 20:48:54.678895 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676181    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 20:48:54.678895 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676184    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 20:48:54.678895 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676187    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 20:48:54.678895 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676189    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 20:48:54.678895 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676194    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 20:48:54.679419 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676197    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 20:48:54.679419 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676200    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 20:48:54.679419 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676203    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 20:48:54.679419 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676205    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 20:48:54.679419 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676208    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 20:48:54.679419 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676210    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 20:48:54.679419 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676213    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 20:48:54.679419 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676216    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 20:48:54.679419 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676218    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 20:48:54.679419 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676221    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 20:48:54.679419 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676223    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 20:48:54.679419 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676226    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 20:48:54.679419 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676228    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 20:48:54.679419 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676231    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 20:48:54.679419 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676233    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 20:48:54.679419 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676236    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 20:48:54.679419 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676238    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 20:48:54.679419 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676241    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 20:48:54.679419 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676243    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 20:48:54.679911 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676246    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 20:48:54.679911 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676248    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 20:48:54.679911 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676251    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 20:48:54.679911 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676254    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 20:48:54.679911 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676258    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 20:48:54.679911 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676260    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 20:48:54.679911 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676263    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 20:48:54.679911 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676265    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 20:48:54.679911 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676268    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 20:48:54.679911 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676271    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 20:48:54.679911 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676273    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 20:48:54.679911 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676277    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 20:48:54.679911 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676280    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 20:48:54.679911 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676282    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 20:48:54.679911 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676285    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 20:48:54.679911 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676288    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 20:48:54.679911 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676290    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 20:48:54.679911 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676292    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 20:48:54.679911 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676295    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 20:48:54.679911 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676297    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 20:48:54.680411 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676300    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 20:48:54.680411 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676303    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 20:48:54.680411 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676305    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 20:48:54.680411 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676308    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 20:48:54.680411 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676310    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 20:48:54.680411 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676313    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 20:48:54.680411 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676315    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 20:48:54.680411 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676318    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 20:48:54.680411 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676320    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 20:48:54.680411 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676323    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 20:48:54.680411 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676325    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 20:48:54.680411 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676328    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 20:48:54.680411 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676330    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 20:48:54.680411 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676333    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 20:48:54.680411 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676335    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 20:48:54.680411 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676338    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 20:48:54.680411 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676341    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 20:48:54.680411 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676345    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 20:48:54.680411 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676349    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 20:48:54.680886 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.676352    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 20:48:54.680886 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676435    2567 flags.go:64] FLAG: --address="0.0.0.0"
Sep 17 20:48:54.680886 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676469    2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 17 20:48:54.680886 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676475    2567 flags.go:64] FLAG: --anonymous-auth="true"
Sep 17 20:48:54.680886 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676480    2567 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 17 20:48:54.680886 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676485    2567 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 17 20:48:54.680886 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676489    2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 17 20:48:54.680886 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676493    2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 17 20:48:54.680886 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676497    2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 17 20:48:54.680886 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676500    2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 17 20:48:54.680886 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676503    2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 17 20:48:54.680886 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676507    2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 17 20:48:54.680886 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676510    2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 17 20:48:54.680886 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676514    2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 17 20:48:54.680886 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676517    2567 flags.go:64] FLAG: --cgroup-root=""
Sep 17 20:48:54.680886 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676520    2567 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 17 20:48:54.680886 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676523    2567 flags.go:64] FLAG: --client-ca-file=""
Sep 17 20:48:54.680886 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676526    2567 flags.go:64] FLAG: --cloud-provider="external"
Sep 17 20:48:54.680886 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676529    2567 flags.go:64] FLAG: --cluster-dns="[]"
Sep 17 20:48:54.680886 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676533    2567 flags.go:64] FLAG: --cluster-domain=""
Sep 17 20:48:54.680886 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676536    2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 17 20:48:54.680886 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676539    2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 17 20:48:54.680886 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676542    2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 17 20:48:54.680886 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676546    2567 flags.go:64] FLAG: --container-log-max-files="5"
Sep 17 20:48:54.681506 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676549    2567 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 17 20:48:54.681506 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676552    2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 17 20:48:54.681506 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676555    2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 17 20:48:54.681506 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676558    2567 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 17 20:48:54.681506 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676562    2567 flags.go:64] FLAG: --contention-profiling="false"
Sep 17 20:48:54.681506 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676565    2567 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 17 20:48:54.681506 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676568    2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 17 20:48:54.681506 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676571    2567 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 17 20:48:54.681506 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676574    2567 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 17 20:48:54.681506 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676578    2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 17 20:48:54.681506 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676581    2567 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 17 20:48:54.681506 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676584    2567 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 17 20:48:54.681506 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676587    2567 flags.go:64] FLAG: --enable-load-reader="false"
Sep 17 20:48:54.681506 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676590    2567 flags.go:64] FLAG: --enable-server="true"
Sep 17 20:48:54.681506 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676594    2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 17 20:48:54.681506 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676598    2567 flags.go:64] FLAG: --event-burst="100"
Sep 17 20:48:54.681506 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676601    2567 flags.go:64] FLAG: --event-qps="50"
Sep 17 20:48:54.681506 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676604    2567 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 17 20:48:54.681506 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676607    2567 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 17 20:48:54.681506 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676610    2567 flags.go:64] FLAG: --eviction-hard=""
Sep 17 20:48:54.681506 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676614    2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 17 20:48:54.681506 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676617    2567 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 17 20:48:54.681506 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676620    2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 17 20:48:54.681506 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676623    2567 flags.go:64] FLAG: --eviction-soft=""
Sep 17 20:48:54.681506 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676626    2567 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 17 20:48:54.682101 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676629    2567 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 17 20:48:54.682101 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676632    2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 17 20:48:54.682101 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676635    2567 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 17 20:48:54.682101 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676637    2567 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 17 20:48:54.682101 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676640    2567 flags.go:64] FLAG: --fail-swap-on="true"
Sep 17 20:48:54.682101 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676643    2567 flags.go:64] FLAG: --feature-gates=""
Sep 17 20:48:54.682101 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676647    2567 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 17 20:48:54.682101 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676650    2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 17 20:48:54.682101 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676653    2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 17 20:48:54.682101 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676656    2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 17 20:48:54.682101 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676659    2567 flags.go:64] FLAG: --healthz-port="10248"
Sep 17 20:48:54.682101 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676662    2567 flags.go:64] FLAG: --help="false"
Sep 17 20:48:54.682101 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676665    2567 flags.go:64] FLAG: --hostname-override="ip-10-0-132-201.ec2.internal"
Sep 17 20:48:54.682101 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676668    2567 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 17 20:48:54.682101 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676672    2567 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 17 20:48:54.682101 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676675    2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 17 20:48:54.682101 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676678    2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 17 20:48:54.682101 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676682    2567 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 17 20:48:54.682101 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676684    2567 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 17 20:48:54.682101 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676687    2567 flags.go:64] FLAG: --image-service-endpoint=""
Sep 17 20:48:54.682101 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676690    2567 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 17 20:48:54.682101 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676693    2567 flags.go:64] FLAG: --kube-api-burst="100"
Sep 17 20:48:54.682101 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676697    2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 17 20:48:54.682734 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676700    2567 flags.go:64] FLAG: --kube-api-qps="50"
Sep 17 20:48:54.682734 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676703    2567 flags.go:64] FLAG: --kube-reserved=""
Sep 17 20:48:54.682734 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676706    2567 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 17 20:48:54.682734 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676709    2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 17 20:48:54.682734 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676713    2567 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 17 20:48:54.682734 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676716    2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 17 20:48:54.682734 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676719    2567 flags.go:64] FLAG: --lock-file=""
Sep 17 20:48:54.682734 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676722    2567 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 17 20:48:54.682734 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676725    2567 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 17 20:48:54.682734 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676728    2567 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 17 20:48:54.682734 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676733    2567 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 17 20:48:54.682734 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676736    2567 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 17 20:48:54.682734 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676739    2567 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 17 20:48:54.682734 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676742    2567 flags.go:64] FLAG: --logging-format="text"
Sep 17 20:48:54.682734 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676745    2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 17 20:48:54.682734 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676748    2567 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 17 20:48:54.682734 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676751    2567 flags.go:64] FLAG: --manifest-url=""
Sep 17 20:48:54.682734 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676754    2567 flags.go:64] FLAG: --manifest-url-header=""
Sep 17 20:48:54.682734 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676758    2567 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 17 20:48:54.682734 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676761    2567 flags.go:64] FLAG: --max-open-files="1000000"
Sep 17 20:48:54.682734 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676765    2567 flags.go:64] FLAG: --max-pods="110"
Sep 17 20:48:54.682734 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676768    2567 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 17 20:48:54.682734 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676771    2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 17 20:48:54.682734 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676774    2567 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 17 20:48:54.682734 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676777    2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 17 20:48:54.683349 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676780    2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 17 20:48:54.683349 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676783    2567 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 17 20:48:54.683349 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676786    2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 17 20:48:54.683349 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676798    2567 flags.go:64] FLAG: --node-status-max-images="50"
Sep 17 20:48:54.683349 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676801    2567 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 17 20:48:54.683349 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676804    2567 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 17 20:48:54.683349 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676807    2567 flags.go:64] FLAG: --pod-cidr=""
Sep 17 20:48:54.683349 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676811    2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 17 20:48:54.683349 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676817    2567 flags.go:64] FLAG: --pod-manifest-path=""
Sep 17 20:48:54.683349 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676820    2567 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 17 20:48:54.683349 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676823    2567 flags.go:64] FLAG: --pods-per-core="0"
Sep 17 20:48:54.683349 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676827    2567 flags.go:64] FLAG: --port="10250"
Sep 17 20:48:54.683349 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676830    2567 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 17 20:48:54.683349 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676833    2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a7178c15ecbb322a"
Sep 17 20:48:54.683349 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676836    2567 flags.go:64] FLAG: --qos-reserved=""
Sep 17 20:48:54.683349 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676839    2567 flags.go:64] FLAG: --read-only-port="10255"
Sep 17 20:48:54.683349 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676842    2567 flags.go:64] FLAG: --register-node="true"
Sep 17 20:48:54.683349 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676845    2567 flags.go:64] FLAG: --register-with-taints=""
Sep 17 20:48:54.683349 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676852    2567 flags.go:64] FLAG: --registry-burst="10"
Sep 17 20:48:54.683349 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676855    2567 flags.go:64] FLAG: --registry-qps="5"
Sep 17 20:48:54.683349 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676858    2567 flags.go:64] FLAG: --reserved-cpus=""
Sep 17 20:48:54.683349 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676861    2567 flags.go:64] FLAG: --reserved-memory=""
Sep 17 20:48:54.683349 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676864    2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 17 20:48:54.683349 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676867    2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 17 20:48:54.683349 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676870    2567 flags.go:64] FLAG: --rotate-certificates="false"
Sep 17 20:48:54.683965 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676873    2567 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 17 20:48:54.683965 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676876    2567 flags.go:64] FLAG: --runonce="false"
Sep 17 20:48:54.683965 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676879    2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 17 20:48:54.683965 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676882    2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 17 20:48:54.683965 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676885    2567 flags.go:64] FLAG: --seccomp-default="false"
Sep 17 20:48:54.683965 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676888    2567 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 17 20:48:54.683965 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676891    2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 17 20:48:54.683965 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676894    2567 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 17 20:48:54.683965 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676897    2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 17 20:48:54.683965 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676900    2567 flags.go:64] FLAG: --storage-driver-password="root"
Sep 17 20:48:54.683965 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676903    2567 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 17 20:48:54.683965 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676906    2567 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 17 20:48:54.683965 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676908    2567 flags.go:64] FLAG: --storage-driver-user="root"
Sep 17 20:48:54.683965 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676911    2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 17 20:48:54.683965 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676914    2567 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 17 20:48:54.683965 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676918    2567 flags.go:64] FLAG: --system-cgroups=""
Sep 17 20:48:54.683965 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676921    2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 17 20:48:54.683965 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676926    2567 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 17 20:48:54.683965 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676930    2567 flags.go:64] FLAG: --tls-cert-file=""
Sep 17 20:48:54.683965 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676933    2567 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 17 20:48:54.683965 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676937    2567 flags.go:64] FLAG: --tls-min-version=""
Sep 17 20:48:54.683965 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676940    2567 flags.go:64] FLAG: --tls-private-key-file=""
Sep 17 20:48:54.683965 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676943    2567 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 17 20:48:54.683965 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676946    2567 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 17 20:48:54.683965 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676949    2567 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 17 20:48:54.684588 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676951    2567 flags.go:64] FLAG: --v="2"
Sep 17 20:48:54.684588 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676955    2567 flags.go:64] FLAG: --version="false"
Sep 17 20:48:54.684588 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676959    2567 flags.go:64] FLAG: --vmodule=""
Sep 17 20:48:54.684588 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676962    2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 17 20:48:54.684588 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.676966    2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 17 20:48:54.684588 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677074    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 20:48:54.684588 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677077    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 20:48:54.684588 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677080    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 20:48:54.684588 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677084    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 20:48:54.684588 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677086    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 20:48:54.684588 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677089    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 20:48:54.684588 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677093    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 20:48:54.684588 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677096    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 20:48:54.684588 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677099    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 20:48:54.684588 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677102    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 20:48:54.684588 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677105    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 20:48:54.684588 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677107    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 20:48:54.684588 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677110    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 20:48:54.684588 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677113    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 20:48:54.684588 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677115    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 20:48:54.684588 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677118    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 20:48:54.685523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677121    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 20:48:54.685523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677123    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 20:48:54.685523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677127    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 20:48:54.685523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677130    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 20:48:54.685523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677134    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 20:48:54.685523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677137    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 20:48:54.685523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677140    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 20:48:54.685523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677142    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 20:48:54.685523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677145    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 20:48:54.685523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677147    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 20:48:54.685523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677150    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 20:48:54.685523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677153    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 20:48:54.685523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677155    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 20:48:54.685523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677158    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 20:48:54.685523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677160    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 20:48:54.685523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677163    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 20:48:54.685523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677166    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 20:48:54.685523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677168    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 20:48:54.685523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677171    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 20:48:54.685523 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677174    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 20:48:54.686399 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677176    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 20:48:54.686399 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677179    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 20:48:54.686399 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677183    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 20:48:54.686399 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677186    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 20:48:54.686399 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677189    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 20:48:54.686399 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677191    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 20:48:54.686399 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677194    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 20:48:54.686399 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677196    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 20:48:54.686399 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677199    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 20:48:54.686399 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677203    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 20:48:54.686399 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677207    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 20:48:54.686399 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677210    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 20:48:54.686399 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677213    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 20:48:54.686399 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677217    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 20:48:54.686399 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677221    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 20:48:54.686399 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677227    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 20:48:54.686399 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677231    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 20:48:54.686399 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677234    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 20:48:54.686399 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677237    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 20:48:54.687143 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677240    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 20:48:54.687143 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677242    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 20:48:54.687143 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677245    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 20:48:54.687143 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677248    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 20:48:54.687143 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677250    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 20:48:54.687143 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677253    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 20:48:54.687143 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677256    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 20:48:54.687143 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677258    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 20:48:54.687143 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677261    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 20:48:54.687143 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677263    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 20:48:54.687143 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677266    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 20:48:54.687143 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677269    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 20:48:54.687143 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677272    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 20:48:54.687143 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677274    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 20:48:54.687143 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677277    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 20:48:54.687143 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677281    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 20:48:54.687143 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677283    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 20:48:54.687143 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677286    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 20:48:54.687143 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677289    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 20:48:54.687143 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677291    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 20:48:54.688254 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677294    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 20:48:54.688254 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677297    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 20:48:54.688254 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677300    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 20:48:54.688254 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677302    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 20:48:54.688254 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677305    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 20:48:54.688254 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677307    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 20:48:54.688254 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677310    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 20:48:54.688254 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677313    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 20:48:54.688254 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677316    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 20:48:54.688254 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677320    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 20:48:54.688254 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677323    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 20:48:54.688254 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677325    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 20:48:54.688254 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677328    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 20:48:54.688254 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677330    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 20:48:54.688254 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677333    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 20:48:54.688254 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677336    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 20:48:54.688254 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677338    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 20:48:54.688254 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677341    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 20:48:54.688254 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677343    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 20:48:54.688254 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677346    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 20:48:54.688883 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677349    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 20:48:54.688883 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677353    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 20:48:54.688883 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677356    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 20:48:54.688883 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.677359    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 20:48:54.688883 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.678265    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 20:48:54.688883 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.686856    2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 17 20:48:54.688883 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.687037    2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 17 20:48:54.688883 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.687061    2567 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 17 20:48:54.688883 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.687074    2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 17 20:48:54.688883 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687156    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 20:48:54.688883 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687163    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 20:48:54.688883 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687176    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 20:48:54.688883 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687181    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 20:48:54.688883 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687186    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 20:48:54.688883 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687190    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 20:48:54.689303 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687195    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 20:48:54.689303 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687200    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 20:48:54.689303 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687204    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 20:48:54.689303 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687208    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 20:48:54.689303 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687212    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 20:48:54.689303 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687217    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 20:48:54.689303 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687221    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 20:48:54.689303 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687225    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 20:48:54.689303 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687229    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 20:48:54.689303 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687233    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 20:48:54.689303 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687238    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 20:48:54.689303 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687242    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 20:48:54.689303 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687247    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 20:48:54.689303 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687251    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 20:48:54.689303 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687256    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 20:48:54.689303 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687260    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 20:48:54.689303 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687265    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 20:48:54.689303 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687269    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 20:48:54.689303 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687273    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 20:48:54.689303 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687278    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 20:48:54.690133 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687282    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 20:48:54.690133 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687286    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 20:48:54.690133 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687291    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 20:48:54.690133 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687297    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 20:48:54.690133 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687302    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 20:48:54.690133 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687306    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 20:48:54.690133 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687310    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 20:48:54.690133 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687315    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 20:48:54.690133 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687319    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 20:48:54.690133 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687323    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 20:48:54.690133 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687327    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 20:48:54.690133 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687332    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 20:48:54.690133 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687336    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 20:48:54.690133 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687340    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 20:48:54.690133 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687344    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 20:48:54.690133 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687349    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 20:48:54.690133 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687353    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 20:48:54.690133 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687357    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 20:48:54.690133 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687361    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 20:48:54.690133 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687385    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 20:48:54.690802 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687389    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 20:48:54.690802 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687393    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 20:48:54.690802 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687397    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 20:48:54.690802 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687401    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 20:48:54.690802 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687404    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 20:48:54.690802 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687408    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 20:48:54.690802 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687412    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 20:48:54.690802 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687418    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 20:48:54.690802 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687423    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 20:48:54.690802 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687428    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 20:48:54.690802 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687432    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 20:48:54.690802 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687437    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 20:48:54.690802 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687443    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 20:48:54.690802 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687449    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 20:48:54.690802 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687453    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 20:48:54.690802 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687458    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 20:48:54.690802 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687463    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 20:48:54.690802 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687467    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 20:48:54.690802 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687472    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 20:48:54.691275 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687475    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 20:48:54.691275 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687482    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 20:48:54.691275 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687488    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 20:48:54.691275 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687493    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 20:48:54.691275 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687498    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 20:48:54.691275 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687502    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 20:48:54.691275 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687507    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 20:48:54.691275 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687512    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 20:48:54.691275 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687517    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 20:48:54.691275 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687521    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 20:48:54.691275 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687525    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 20:48:54.691275 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687529    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 20:48:54.691275 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687534    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 20:48:54.691275 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687538    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 20:48:54.691275 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687543    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 20:48:54.691275 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687547    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 20:48:54.691275 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687551    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 20:48:54.691275 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687555    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 20:48:54.691275 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687559    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 20:48:54.691763 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687563    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 20:48:54.691763 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687568    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 20:48:54.691763 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687572    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 20:48:54.691763 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687576    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 20:48:54.691763 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687581    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 20:48:54.691763 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687585    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 20:48:54.691763 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687589    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 20:48:54.691763 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687594    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 20:48:54.691763 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687598    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 20:48:54.691763 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687603    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 20:48:54.691763 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687607    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 20:48:54.691763 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687612    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 20:48:54.691763 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687616    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 20:48:54.691763 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687621    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 20:48:54.691763 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687625    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 20:48:54.692128 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.687634    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 20:48:54.692128 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687812    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 20:48:54.692128 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687819    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 20:48:54.692128 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687824    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 20:48:54.692128 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687829    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 20:48:54.692128 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687834    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 20:48:54.692128 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687838    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 20:48:54.692128 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687843    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 20:48:54.692128 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687847    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 20:48:54.692128 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687851    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 20:48:54.692128 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687855    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 20:48:54.692128 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687859    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 20:48:54.692128 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687864    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 20:48:54.692128 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687868    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 20:48:54.692128 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687872    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 20:48:54.692597 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687877    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 20:48:54.692597 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687881    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 20:48:54.692597 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687885    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 20:48:54.692597 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687889    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 20:48:54.692597 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687893    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 20:48:54.692597 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687897    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 20:48:54.692597 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687902    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 20:48:54.692597 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687906    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 20:48:54.692597 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687910    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 20:48:54.692597 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687914    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 20:48:54.692597 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687918    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 20:48:54.692597 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687923    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 20:48:54.692597 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687927    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 20:48:54.692597 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687931    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 20:48:54.692597 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687936    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 20:48:54.692597 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687940    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 20:48:54.692597 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687945    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 20:48:54.692597 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687949    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 20:48:54.692597 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687953    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 20:48:54.692597 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687957    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 20:48:54.693084 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687960    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 20:48:54.693084 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687964    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 20:48:54.693084 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687967    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 20:48:54.693084 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687973    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 20:48:54.693084 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687979    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 20:48:54.693084 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687983    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 20:48:54.693084 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687987    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 20:48:54.693084 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687992    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 20:48:54.693084 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.687996    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 20:48:54.693084 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688001    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 20:48:54.693084 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688005    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 20:48:54.693084 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688009    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 20:48:54.693084 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688013    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 20:48:54.693084 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688018    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 20:48:54.693084 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688021    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 20:48:54.693084 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688025    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 20:48:54.693084 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688029    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 20:48:54.693084 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688033    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 20:48:54.693084 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688037    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 20:48:54.693084 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688041    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 20:48:54.693589 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688045    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 20:48:54.693589 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688049    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 20:48:54.693589 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688054    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 20:48:54.693589 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688058    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 20:48:54.693589 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688063    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 20:48:54.693589 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688067    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 20:48:54.693589 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688071    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 20:48:54.693589 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688077    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 20:48:54.693589 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688081    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 20:48:54.693589 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688085    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 20:48:54.693589 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688090    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 20:48:54.693589 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688094    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 20:48:54.693589 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688099    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 20:48:54.693589 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688103    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 20:48:54.693589 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688107    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 20:48:54.693589 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688111    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 20:48:54.693589 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688116    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 20:48:54.693589 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688120    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 20:48:54.693589 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688124    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 20:48:54.694050 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688128    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 20:48:54.694050 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688132    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 20:48:54.694050 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688137    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 20:48:54.694050 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688141    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 20:48:54.694050 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688146    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 20:48:54.694050 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688150    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 20:48:54.694050 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688154    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 20:48:54.694050 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688159    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 20:48:54.694050 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688163    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 20:48:54.694050 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688167    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 20:48:54.694050 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688171    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 20:48:54.694050 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688175    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 20:48:54.694050 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688180    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 20:48:54.694050 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688184    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 20:48:54.694050 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688188    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 20:48:54.694050 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688193    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 20:48:54.694050 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688197    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 20:48:54.694050 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688201    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 20:48:54.694050 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688205    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 20:48:54.694050 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688209    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 20:48:54.694548 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688214    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 20:48:54.694548 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688220    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 20:48:54.694548 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688226    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 20:48:54.694548 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688231    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 20:48:54.694548 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688237    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 20:48:54.694548 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:54.688243    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 20:48:54.694548 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.688251    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 20:48:54.694548 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.688418    2567 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 17 20:48:54.694548 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.694279    2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 17 20:48:54.695345 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.695333    2567 server.go:1019] "Starting client certificate rotation"
Sep 17 20:48:54.695418 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.695401    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 20:48:54.695448 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.695433    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 20:48:54.698200 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.698187    2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 20:48:54.703402 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.703388    2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 20:48:54.716004 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.715986    2567 log.go:25] "Validated CRI v1 runtime API"
Sep 17 20:48:54.718087 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.718072    2567 log.go:25] "Validated CRI v1 image API"
Sep 17 20:48:54.719004 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.718987    2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 17 20:48:54.725692 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.725675    2567 fs.go:135] Filesystem UUIDs: map[35b846f2-fc42-458d-83b8-90e5ef0a20e7:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 b88277b6-c899-4b95-baa8-d9d53afabdc7:/dev/nvme0n1p4]
Sep 17 20:48:54.725758 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.725692    2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 17 20:48:54.731835 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.731715    2567 manager.go:216] Machine: {Timestamp:2026-09-17 20:48:54.730303517 +0000 UTC m=+0.373293407 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101582 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec224cf4fc61675a6013f9fb44f48244 SystemUUID:ec224cf4-fc61-675a-6013-f9fb44f48244 BootID:e9e0cc7e-52c6-4d15-a249-0d9bf3080f4c Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:eb:26:71:f1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:eb:26:71:f1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:02:d6:55:cb:1f:93 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 17 20:48:54.731835 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.731823    2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 17 20:48:54.731956 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.731916    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 20:48:54.731956 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.731946    2567 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 17 20:48:54.736286 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.736162    2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 17 20:48:54.736703 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.736349    2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-201.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 17 20:48:54.738182 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.738167    2567 topology_manager.go:138] "Creating topology manager with none policy"
Sep 17 20:48:54.738252 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.738189    2567 container_manager_linux.go:309] "Creating device plugin manager"
Sep 17 20:48:54.738252 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.738200    2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 20:48:54.739000 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.738988    2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 20:48:54.739086 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.739074    2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 17 20:48:54.743790 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.743776    2567 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 20:48:54.743919 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.743909    2567 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 17 20:48:54.743974 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.743948    2567 kubelet.go:486] "Attempting to sync node with API server"
Sep 17 20:48:54.743974 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.743963    2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 17 20:48:54.744065 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.743983    2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 17 20:48:54.744065 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.743996    2567 kubelet.go:398] "Adding apiserver pod source"
Sep 17 20:48:54.744065 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.744009    2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 17 20:48:54.744393 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.744365    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 20:48:54.744458 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.744408    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 20:48:54.746734 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.746720    2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 17 20:48:54.749815 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.749799    2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 17 20:48:54.749885 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.749831    2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 17 20:48:54.752055 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.752043    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 17 20:48:54.752102 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.752060    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 17 20:48:54.752102 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.752066    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 17 20:48:54.752102 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.752071    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 17 20:48:54.752102 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.752079    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 17 20:48:54.752102 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.752086    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 17 20:48:54.752102 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.752099    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 17 20:48:54.752102 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.752104    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 17 20:48:54.752325 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.752110    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 17 20:48:54.752325 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.752115    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 17 20:48:54.752325 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.752123    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 17 20:48:54.752325 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.752130    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 17 20:48:54.753945 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.753922    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 17 20:48:54.753945 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.753940    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 17 20:48:54.754940 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:54.754918    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 17 20:48:54.755019 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:54.754993    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-201.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 17 20:48:54.757764 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.757752    2567 server.go:1268] "Started kubelet"
Sep 17 20:48:54.757873 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.757839    2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 17 20:48:54.757971 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.757838    2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 17 20:48:54.757971 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.757909    2567 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 17 20:48:54.758573 ip-10-0-132-201 systemd[1]: Started Kubernetes Kubelet.
Sep 17 20:48:54.759363 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.759347    2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 17 20:48:54.761453 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.761437    2567 server.go:310] "Adding debug handlers to kubelet server"
Sep 17 20:48:54.762819 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:54.762754    2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 17 20:48:54.766009 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.765992    2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-201.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 17 20:48:54.767328 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:54.766196    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-201.ec2.internal.18d636c80c8480df  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-201.ec2.internal,UID:ip-10-0-132-201.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-201.ec2.internal,},FirstTimestamp:2026-09-17 20:48:54.757720287 +0000 UTC m=+0.400710157,LastTimestamp:2026-09-17 20:48:54.757720287 +0000 UTC m=+0.400710157,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-201.ec2.internal,}"
Sep 17 20:48:54.768186 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.768171    2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 17 20:48:54.768237 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.768192    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 17 20:48:54.768289 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:54.768269    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-201.ec2.internal\" not found"
Sep 17 20:48:54.769002 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.768987    2567 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 17 20:48:54.769135 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.769118    2567 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 17 20:48:54.769236 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.769033    2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 17 20:48:54.769285 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.769258    2567 reconstruct.go:97] "Volume reconstruction finished"
Sep 17 20:48:54.769285 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.769264    2567 reconciler.go:29] "Reconciler: start to sync state"
Sep 17 20:48:54.770092 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.770078    2567 factory.go:55] Registering systemd factory
Sep 17 20:48:54.770220 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.770210    2567 factory.go:223] Registration of the systemd container factory successfully
Sep 17 20:48:54.770746 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.770707    2567 factory.go:153] Registering CRI-O factory
Sep 17 20:48:54.770746 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.770729    2567 factory.go:223] Registration of the crio container factory successfully
Sep 17 20:48:54.770881 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.770787    2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 17 20:48:54.770881 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.770807    2567 factory.go:103] Registering Raw factory
Sep 17 20:48:54.770881 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.770831    2567 manager.go:1196] Started watching for new ooms in manager
Sep 17 20:48:54.771299 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.771269    2567 manager.go:349] Starting recovery of all containers
Sep 17 20:48:54.773486 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:54.772684    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 17 20:48:54.773795 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:54.773760    2567 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-201.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 17 20:48:54.784241 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.784227    2567 manager.go:354] Recovery completed
Sep 17 20:48:54.785786 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:54.785762    2567 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Sep 17 20:48:54.788964 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.788950    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 20:48:54.790614 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.790594    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-201.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 20:48:54.790701 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.790633    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-201.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 20:48:54.790701 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.790648    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-201.ec2.internal" event="NodeHasSufficientPID"
Sep 17 20:48:54.791196 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.791182    2567 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 17 20:48:54.791266 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.791195    2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 17 20:48:54.791266 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.791216    2567 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 20:48:54.791266 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.791247    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 20:48:54.791944 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.791924    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-201.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 20:48:54.792035 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.791963    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-201.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 20:48:54.792035 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.791979    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-201.ec2.internal" event="NodeHasSufficientPID"
Sep 17 20:48:54.793039 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:54.792947    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-201.ec2.internal.18d636c80e7a713d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-201.ec2.internal,UID:ip-10-0-132-201.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-201.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-201.ec2.internal,},FirstTimestamp:2026-09-17 20:48:54.790615357 +0000 UTC m=+0.433605236,LastTimestamp:2026-09-17 20:48:54.790615357 +0000 UTC m=+0.433605236,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-201.ec2.internal,}"
Sep 17 20:48:54.793457 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.793444    2567 policy_none.go:49] "None policy: Start"
Sep 17 20:48:54.793519 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.793462    2567 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 17 20:48:54.793519 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.793476    2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 17 20:48:54.795534 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.795514    2567 policy_none.go:47] "Start"
Sep 17 20:48:54.796069 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.796054    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-775z4"
Sep 17 20:48:54.800522 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:54.800444    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-201.ec2.internal.18d636c80e7ad0bc  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-201.ec2.internal,UID:ip-10-0-132-201.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-201.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-201.ec2.internal,},FirstTimestamp:2026-09-17 20:48:54.790639804 +0000 UTC m=+0.433629681,LastTimestamp:2026-09-17 20:48:54.790639804 +0000 UTC m=+0.433629681,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-201.ec2.internal,}"
Sep 17 20:48:54.803593 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:54.803503    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-201.ec2.internal.18d636c80e7b0700  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-201.ec2.internal,UID:ip-10-0-132-201.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-132-201.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-132-201.ec2.internal,},FirstTimestamp:2026-09-17 20:48:54.790653696 +0000 UTC m=+0.433643570,LastTimestamp:2026-09-17 20:48:54.790653696 +0000 UTC m=+0.433643570,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-201.ec2.internal,}"
Sep 17 20:48:54.807604 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.807585    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-775z4"
Sep 17 20:48:54.812237 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:54.812166    2567 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-132-201.ec2.internal.18d636c80e7a713d\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-201.ec2.internal.18d636c80e7a713d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-201.ec2.internal,UID:ip-10-0-132-201.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-201.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-201.ec2.internal,},FirstTimestamp:2026-09-17 20:48:54.790615357 +0000 UTC m=+0.433605236,LastTimestamp:2026-09-17 20:48:54.791944425 +0000 UTC m=+0.434934300,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-201.ec2.internal,}"
Sep 17 20:48:54.824082 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.824068    2567 manager.go:337] "Starting Device Plugin manager"
Sep 17 20:48:54.844862 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:54.824107    2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 17 20:48:54.844862 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.824119    2567 server.go:87] "Starting device plugin registration server"
Sep 17 20:48:54.844862 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.824418    2567 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 17 20:48:54.844862 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.824431    2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 17 20:48:54.844862 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.824513    2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 17 20:48:54.844862 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.824589    2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 17 20:48:54.844862 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.824598    2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 17 20:48:54.844862 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:54.825409    2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 17 20:48:54.844862 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:54.825445    2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-201.ec2.internal\" not found"
Sep 17 20:48:54.864993 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.864964    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 17 20:48:54.866135 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.866115    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 17 20:48:54.866221 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.866157    2567 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 17 20:48:54.866221 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.866180    2567 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 17 20:48:54.866221 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:54.866215    2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 17 20:48:54.868303 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.868285    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 20:48:54.925204 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.925122    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 20:48:54.926167 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.926152    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-201.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 20:48:54.926270 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.926193    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-201.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 20:48:54.926270 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.926208    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-201.ec2.internal" event="NodeHasSufficientPID"
Sep 17 20:48:54.926270 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.926238    2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-201.ec2.internal"
Sep 17 20:48:54.933961 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.933945    2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-201.ec2.internal"
Sep 17 20:48:54.934038 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:54.933967    2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-201.ec2.internal\": node \"ip-10-0-132-201.ec2.internal\" not found"
Sep 17 20:48:54.952482 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:54.952465    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-201.ec2.internal\" not found"
Sep 17 20:48:54.967321 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.967302    2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-201.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-201.ec2.internal"]
Sep 17 20:48:54.967381 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.967361    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 20:48:54.968738 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.968720    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-201.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 20:48:54.968828 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.968746    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-201.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 20:48:54.968828 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.968756    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-201.ec2.internal" event="NodeHasSufficientPID"
Sep 17 20:48:54.969917 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.969897    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b0dbf9b7758dc823cc6b6f466c3213a7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-201.ec2.internal\" (UID: \"b0dbf9b7758dc823cc6b6f466c3213a7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-201.ec2.internal"
Sep 17 20:48:54.970014 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.969921    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0dbf9b7758dc823cc6b6f466c3213a7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-201.ec2.internal\" (UID: \"b0dbf9b7758dc823cc6b6f466c3213a7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-201.ec2.internal"
Sep 17 20:48:54.971087 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.971076    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 20:48:54.971205 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.971190    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-201.ec2.internal"
Sep 17 20:48:54.971244 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.971218    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 20:48:54.971773 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.971756    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-201.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 20:48:54.971773 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.971768    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-201.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 20:48:54.971888 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.971785    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-201.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 20:48:54.971888 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.971787    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-201.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 20:48:54.971888 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.971798    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-201.ec2.internal" event="NodeHasSufficientPID"
Sep 17 20:48:54.971888 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.971807    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-201.ec2.internal" event="NodeHasSufficientPID"
Sep 17 20:48:54.974043 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.974029    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-201.ec2.internal"
Sep 17 20:48:54.974093 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.974058    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 20:48:54.974698 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.974683    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-201.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 20:48:54.974787 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.974708    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-201.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 20:48:54.974787 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:54.974723    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-201.ec2.internal" event="NodeHasSufficientPID"
Sep 17 20:48:54.992969 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:54.992949    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-201.ec2.internal\" not found" node="ip-10-0-132-201.ec2.internal"
Sep 17 20:48:54.997413 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:54.997396    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-201.ec2.internal\" not found" node="ip-10-0-132-201.ec2.internal"
Sep 17 20:48:55.052660 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:55.052639    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-201.ec2.internal\" not found"
Sep 17 20:48:55.070479 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:55.070452    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b0dbf9b7758dc823cc6b6f466c3213a7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-201.ec2.internal\" (UID: \"b0dbf9b7758dc823cc6b6f466c3213a7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-201.ec2.internal"
Sep 17 20:48:55.070611 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:55.070479    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0dbf9b7758dc823cc6b6f466c3213a7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-201.ec2.internal\" (UID: \"b0dbf9b7758dc823cc6b6f466c3213a7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-201.ec2.internal"
Sep 17 20:48:55.070611 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:55.070512    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bf877a7e2526fe3fd4e830b9589fdf4b-config\") pod \"kube-apiserver-proxy-ip-10-0-132-201.ec2.internal\" (UID: \"bf877a7e2526fe3fd4e830b9589fdf4b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-201.ec2.internal"
Sep 17 20:48:55.070611 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:55.070546    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b0dbf9b7758dc823cc6b6f466c3213a7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-201.ec2.internal\" (UID: \"b0dbf9b7758dc823cc6b6f466c3213a7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-201.ec2.internal"
Sep 17 20:48:55.070611 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:55.070559    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0dbf9b7758dc823cc6b6f466c3213a7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-201.ec2.internal\" (UID: \"b0dbf9b7758dc823cc6b6f466c3213a7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-201.ec2.internal"
Sep 17 20:48:55.153166 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:55.153134    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-201.ec2.internal\" not found"
Sep 17 20:48:55.171476 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:55.171454    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bf877a7e2526fe3fd4e830b9589fdf4b-config\") pod \"kube-apiserver-proxy-ip-10-0-132-201.ec2.internal\" (UID: \"bf877a7e2526fe3fd4e830b9589fdf4b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-201.ec2.internal"
Sep 17 20:48:55.171476 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:55.171470    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bf877a7e2526fe3fd4e830b9589fdf4b-config\") pod \"kube-apiserver-proxy-ip-10-0-132-201.ec2.internal\" (UID: \"bf877a7e2526fe3fd4e830b9589fdf4b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-201.ec2.internal"
Sep 17 20:48:56.551202 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:56.551181    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-201.ec2.internal\" not found"
Sep 17 20:48:56.594688 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:56.594657    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-201.ec2.internal"
Sep 17 20:48:56.598987 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:56.598968    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-201.ec2.internal"
Sep 17 20:48:56.651541 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:56.651509    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-201.ec2.internal\" not found"
Sep 17 20:48:56.752026 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:56.751993    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-201.ec2.internal\" not found"
Sep 17 20:48:56.852609 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:56.852534    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-201.ec2.internal\" not found"
Sep 17 20:48:56.953122 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:56.953099    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-201.ec2.internal\" not found"
Sep 17 20:48:56.977529 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:56.977510    2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 20:48:56.992144 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:56.992127    2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 17 20:48:56.992242 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:56.992231    2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 17 20:48:56.992295 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:56.992271    2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 17 20:48:57.053610 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:57.053573    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-201.ec2.internal\" not found"
Sep 17 20:48:57.111712 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:57.111638    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-16 20:43:56 +0000 UTC" deadline="2028-04-20 20:40:47.436270842 +0000 UTC"
Sep 17 20:48:57.111712 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:57.111670    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13943h51m50.324604538s"
Sep 17 20:48:57.117807 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:57.117788    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 20:48:57.118690 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:57.118672    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-201.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 20:48:57.118779 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:57.118709    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-201.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 20:48:57.118779 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:57.118725    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-201.ec2.internal" event="NodeHasSufficientPID"
Sep 17 20:48:57.154431 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:57.154406    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-201.ec2.internal\" not found"
Sep 17 20:48:57.177510 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:57.177490    2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 20:48:57.255227 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:57.255005    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-201.ec2.internal\" not found"
Sep 17 20:48:57.337083 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:57.336769    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0dbf9b7758dc823cc6b6f466c3213a7.slice/crio-ded08b82ef457eecdbbc71c681c47a475a12133198483e8d33622108611c9281 WatchSource:0}: Error finding container ded08b82ef457eecdbbc71c681c47a475a12133198483e8d33622108611c9281: Status 404 returned error can't find the container with id ded08b82ef457eecdbbc71c681c47a475a12133198483e8d33622108611c9281
Sep 17 20:48:57.337207 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:57.337142    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf877a7e2526fe3fd4e830b9589fdf4b.slice/crio-f681fc27541516b95f08585ac3060d08c200576567495064fff98d5083575bb4 WatchSource:0}: Error finding container f681fc27541516b95f08585ac3060d08c200576567495064fff98d5083575bb4: Status 404 returned error can't find the container with id f681fc27541516b95f08585ac3060d08c200576567495064fff98d5083575bb4
Sep 17 20:48:57.342954 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:57.342934    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 20:48:57.355902 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:57.355878    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-201.ec2.internal\" not found"
Sep 17 20:48:57.456492 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:57.456411    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-201.ec2.internal\" not found"
Sep 17 20:48:57.472935 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:57.472914    2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 20:48:57.566524 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:57.566499    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-201.ec2.internal"
Sep 17 20:48:57.579968 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:57.579947    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 20:48:57.580943 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:57.580930    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-201.ec2.internal"
Sep 17 20:48:57.590524 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:57.590508    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 20:48:58.043034 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.042992    2567 apiserver.go:52] "Watching apiserver"
Sep 17 20:48:58.051677 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.051648    2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 17 20:48:58.052111 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.052080    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-201.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5rb9","openshift-image-registry/node-ca-6h956","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-201.ec2.internal","openshift-multus/multus-additional-cni-plugins-zt8xk","openshift-multus/network-metrics-daemon-jhhl5","openshift-network-diagnostics/network-check-target-rb4d7","openshift-network-operator/iptables-alerter-llmv2","kube-system/global-pull-secret-syncer-r7jbg","kube-system/konnectivity-agent-4j6w6","openshift-cluster-node-tuning-operator/tuned-s9k9h","openshift-dns/node-resolver-prdg7","openshift-multus/multus-4bpjd","openshift-ovn-kubernetes/ovnkube-node-gcr8f"]
Sep 17 20:48:58.054612 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.054591    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4j6w6"
Sep 17 20:48:58.057599 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.057580    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.058411 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.058389    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 17 20:48:58.058641 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.058623    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 17 20:48:58.059209 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.058829    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-76sgp\""
Sep 17 20:48:58.059209 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.059148    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6h956"
Sep 17 20:48:58.060000 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.059981    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-86lvv\""
Sep 17 20:48:58.060321 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.060286    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 17 20:48:58.060455 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.060438    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 17 20:48:58.060704 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.060687    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 17 20:48:58.060920 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.060903    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 17 20:48:58.061378 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.061357    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 17 20:48:58.062179 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.061610    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-sjxdf\""
Sep 17 20:48:58.062179 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.061615    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.062179 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.061844    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 17 20:48:58.062179 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.061869    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 17 20:48:58.064095 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.064073    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zt8xk"
Sep 17 20:48:58.065046 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.064911    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 17 20:48:58.065046 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.064953    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 17 20:48:58.065978 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.065333    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 17 20:48:58.065978 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.065408    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 17 20:48:58.065978 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.065605    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-s4cm7\""
Sep 17 20:48:58.065978 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.065644    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 17 20:48:58.065978 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.065839    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 17 20:48:58.066320 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.066088    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:48:58.066320 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:58.066146    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhhl5" podUID="b4f3e778-0cec-45fe-be1b-a8789d63c1d7"
Sep 17 20:48:58.071302 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.071282    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fd4tf\""
Sep 17 20:48:58.079177 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.071506    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 17 20:48:58.079177 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.071697    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 17 20:48:58.079177 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.073424    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 17 20:48:58.079177 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.073870    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:48:58.079177 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:58.073970    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r7jbg" podUID="082f604e-de6c-43c3-bd30-02318c0e01a4"
Sep 17 20:48:58.079177 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.074259    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:48:58.079177 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:58.074337    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rb4d7" podUID="c6f7a65c-e785-4e19-aed2-5c2583881c06"
Sep 17 20:48:58.079177 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.074465    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5rb9"
Sep 17 20:48:58.079177 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.077195    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-system-cni-dir\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.079177 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.077233    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-host-var-lib-kubelet\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.079177 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.077275    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-multus-conf-dir\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.079177 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.077306    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1dec440-9d21-409e-a87b-7de07d399656-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zt8xk\" (UID: \"b1dec440-9d21-409e-a87b-7de07d399656\") " pod="openshift-multus/multus-additional-cni-plugins-zt8xk"
Sep 17 20:48:58.079177 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.077316    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 17 20:48:58.079177 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.077326    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-llmv2"
Sep 17 20:48:58.079177 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.077336    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a5fe1eef-ed7f-416d-9ff9-c790010c3024-socket-dir\") pod \"aws-ebs-csi-driver-node-c5rb9\" (UID: \"a5fe1eef-ed7f-416d-9ff9-c790010c3024\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5rb9"
Sep 17 20:48:58.079177 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.077366    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a5fe1eef-ed7f-416d-9ff9-c790010c3024-etc-selinux\") pod \"aws-ebs-csi-driver-node-c5rb9\" (UID: \"a5fe1eef-ed7f-416d-9ff9-c790010c3024\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5rb9"
Sep 17 20:48:58.079177 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.077394    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/447ec589-f25d-47fb-9713-519a04467695-host-slash\") pod \"iptables-alerter-llmv2\" (UID: \"447ec589-f25d-47fb-9713-519a04467695\") " pod="openshift-network-operator/iptables-alerter-llmv2"
Sep 17 20:48:58.079177 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.077422    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-host-run-k8s-cni-cncf-io\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.079177 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.077456    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a5fe1eef-ed7f-416d-9ff9-c790010c3024-sys-fs\") pod \"aws-ebs-csi-driver-node-c5rb9\" (UID: \"a5fe1eef-ed7f-416d-9ff9-c790010c3024\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5rb9"
Sep 17 20:48:58.079177 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.077481    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-var-lib-openvswitch\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.080486 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.077508    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d17553b3-5be3-4200-adec-f293aa7a4f81-ovnkube-config\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.080486 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.077542    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/082f604e-de6c-43c3-bd30-02318c0e01a4-kubelet-config\") pod \"global-pull-secret-syncer-r7jbg\" (UID: \"082f604e-de6c-43c3-bd30-02318c0e01a4\") " pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:48:58.080486 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.077573    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-host-var-lib-cni-bin\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.080486 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.077598    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-run-systemd\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.080486 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.077630    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/082f604e-de6c-43c3-bd30-02318c0e01a4-original-pull-secret\") pod \"global-pull-secret-syncer-r7jbg\" (UID: \"082f604e-de6c-43c3-bd30-02318c0e01a4\") " pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:48:58.080486 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.077658    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-cnibin\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.080486 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.077688    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.080486 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.077720    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/082f604e-de6c-43c3-bd30-02318c0e01a4-dbus\") pod \"global-pull-secret-syncer-r7jbg\" (UID: \"082f604e-de6c-43c3-bd30-02318c0e01a4\") " pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:48:58.080486 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.078070    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 17 20:48:58.080486 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.077745    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-multus-cni-dir\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.080486 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.078505    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gstzd\" (UniqueName: \"kubernetes.io/projected/447ec589-f25d-47fb-9713-519a04467695-kube-api-access-gstzd\") pod \"iptables-alerter-llmv2\" (UID: \"447ec589-f25d-47fb-9713-519a04467695\") " pod="openshift-network-operator/iptables-alerter-llmv2"
Sep 17 20:48:58.080486 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.078538    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d17553b3-5be3-4200-adec-f293aa7a4f81-ovnkube-script-lib\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.080486 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.078568    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6bc4ca39-a164-4ff7-9c1c-fedf495ccb5c-serviceca\") pod \"node-ca-6h956\" (UID: \"6bc4ca39-a164-4ff7-9c1c-fedf495ccb5c\") " pod="openshift-image-registry/node-ca-6h956"
Sep 17 20:48:58.080486 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.078608    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55shf\" (UniqueName: \"kubernetes.io/projected/6bc4ca39-a164-4ff7-9c1c-fedf495ccb5c-kube-api-access-55shf\") pod \"node-ca-6h956\" (UID: \"6bc4ca39-a164-4ff7-9c1c-fedf495ccb5c\") " pod="openshift-image-registry/node-ca-6h956"
Sep 17 20:48:58.080486 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.078633    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-node-log\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.080486 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.078665    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7mbv\" (UniqueName: \"kubernetes.io/projected/c6f7a65c-e785-4e19-aed2-5c2583881c06-kube-api-access-b7mbv\") pod \"network-check-target-rb4d7\" (UID: \"c6f7a65c-e785-4e19-aed2-5c2583881c06\") " pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:48:58.080486 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.078717    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/937db66c-5b56-482e-8402-4fd3fe1d388c-konnectivity-ca\") pod \"konnectivity-agent-4j6w6\" (UID: \"937db66c-5b56-482e-8402-4fd3fe1d388c\") " pod="kube-system/konnectivity-agent-4j6w6"
Sep 17 20:48:58.081293 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.078748    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-run-ovn\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.081293 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.078769    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 17 20:48:58.081293 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.078795    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-log-socket\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.081293 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.078821    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-host-cni-bin\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.081293 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.078851    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1dec440-9d21-409e-a87b-7de07d399656-system-cni-dir\") pod \"multus-additional-cni-plugins-zt8xk\" (UID: \"b1dec440-9d21-409e-a87b-7de07d399656\") " pod="openshift-multus/multus-additional-cni-plugins-zt8xk"
Sep 17 20:48:58.081293 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.078883    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a5fe1eef-ed7f-416d-9ff9-c790010c3024-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c5rb9\" (UID: \"a5fe1eef-ed7f-416d-9ff9-c790010c3024\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5rb9"
Sep 17 20:48:58.081293 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.078912    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-run-openvswitch\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.081293 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.078964    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d17553b3-5be3-4200-adec-f293aa7a4f81-env-overrides\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.081293 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.079091    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-host-run-netns\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.081293 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.079119    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-host-var-lib-cni-multus\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.081293 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.079157    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-host-run-ovn-kubernetes\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.081293 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.079186    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-multus-socket-dir-parent\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.081293 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.079221    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4wzl\" (UniqueName: \"kubernetes.io/projected/a5fe1eef-ed7f-416d-9ff9-c790010c3024-kube-api-access-v4wzl\") pod \"aws-ebs-csi-driver-node-c5rb9\" (UID: \"a5fe1eef-ed7f-416d-9ff9-c790010c3024\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5rb9"
Sep 17 20:48:58.081293 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.079251    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-host-run-netns\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.081293 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.079279    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-etc-kubernetes\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.081293 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.079304    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b1dec440-9d21-409e-a87b-7de07d399656-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zt8xk\" (UID: \"b1dec440-9d21-409e-a87b-7de07d399656\") " pod="openshift-multus/multus-additional-cni-plugins-zt8xk"
Sep 17 20:48:58.081293 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.079335    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a5fe1eef-ed7f-416d-9ff9-c790010c3024-device-dir\") pod \"aws-ebs-csi-driver-node-c5rb9\" (UID: \"a5fe1eef-ed7f-416d-9ff9-c790010c3024\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5rb9"
Sep 17 20:48:58.081999 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.079364    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-systemd-units\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.081999 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.079413    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-host-cni-netd\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.081999 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.079467    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvzp6\" (UniqueName: \"kubernetes.io/projected/d17553b3-5be3-4200-adec-f293aa7a4f81-kube-api-access-bvzp6\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.081999 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.079497    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-hostroot\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.081999 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.079549    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szjxp\" (UniqueName: \"kubernetes.io/projected/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-kube-api-access-szjxp\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.081999 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.079590    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b1dec440-9d21-409e-a87b-7de07d399656-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zt8xk\" (UID: \"b1dec440-9d21-409e-a87b-7de07d399656\") " pod="openshift-multus/multus-additional-cni-plugins-zt8xk"
Sep 17 20:48:58.081999 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.079643    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/937db66c-5b56-482e-8402-4fd3fe1d388c-agent-certs\") pod \"konnectivity-agent-4j6w6\" (UID: \"937db66c-5b56-482e-8402-4fd3fe1d388c\") " pod="kube-system/konnectivity-agent-4j6w6"
Sep 17 20:48:58.081999 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.079683    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d17553b3-5be3-4200-adec-f293aa7a4f81-ovn-node-metrics-cert\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.081999 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.079741    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-host-run-multus-certs\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.081999 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.079799    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgcv5\" (UniqueName: \"kubernetes.io/projected/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-kube-api-access-fgcv5\") pod \"network-metrics-daemon-jhhl5\" (UID: \"b4f3e778-0cec-45fe-be1b-a8789d63c1d7\") " pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:48:58.081999 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.079832    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-multus-daemon-config\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.082745 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.082633    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.082859 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.082801    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a5fe1eef-ed7f-416d-9ff9-c790010c3024-registration-dir\") pod \"aws-ebs-csi-driver-node-c5rb9\" (UID: \"a5fe1eef-ed7f-416d-9ff9-c790010c3024\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5rb9"
Sep 17 20:48:58.082928 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.082844    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-host-kubelet\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.082928 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.082892    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-etc-openvswitch\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.083035 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.082935    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-cni-binary-copy\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.083035 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.082964    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1dec440-9d21-409e-a87b-7de07d399656-cnibin\") pod \"multus-additional-cni-plugins-zt8xk\" (UID: \"b1dec440-9d21-409e-a87b-7de07d399656\") " pod="openshift-multus/multus-additional-cni-plugins-zt8xk"
Sep 17 20:48:58.083035 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.082998    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/447ec589-f25d-47fb-9713-519a04467695-iptables-alerter-script\") pod \"iptables-alerter-llmv2\" (UID: \"447ec589-f25d-47fb-9713-519a04467695\") " pod="openshift-network-operator/iptables-alerter-llmv2"
Sep 17 20:48:58.083035 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.083029    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-host-slash\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.083231 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.083059    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6bc4ca39-a164-4ff7-9c1c-fedf495ccb5c-host\") pod \"node-ca-6h956\" (UID: \"6bc4ca39-a164-4ff7-9c1c-fedf495ccb5c\") " pod="openshift-image-registry/node-ca-6h956"
Sep 17 20:48:58.083231 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.083090    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1dec440-9d21-409e-a87b-7de07d399656-cni-binary-copy\") pod \"multus-additional-cni-plugins-zt8xk\" (UID: \"b1dec440-9d21-409e-a87b-7de07d399656\") " pod="openshift-multus/multus-additional-cni-plugins-zt8xk"
Sep 17 20:48:58.083231 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.083118    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxrhq\" (UniqueName: \"kubernetes.io/projected/b1dec440-9d21-409e-a87b-7de07d399656-kube-api-access-gxrhq\") pod \"multus-additional-cni-plugins-zt8xk\" (UID: \"b1dec440-9d21-409e-a87b-7de07d399656\") " pod="openshift-multus/multus-additional-cni-plugins-zt8xk"
Sep 17 20:48:58.083231 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.083150    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-metrics-certs\") pod \"network-metrics-daemon-jhhl5\" (UID: \"b4f3e778-0cec-45fe-be1b-a8789d63c1d7\") " pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:48:58.083415 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.083386    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-v4x6q\""
Sep 17 20:48:58.084142 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.083912    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 17 20:48:58.084142 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.084062    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 17 20:48:58.084611 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.084365    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 17 20:48:58.084611 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.084434    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-f78qn\""
Sep 17 20:48:58.086258 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.085491    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tmm7f\""
Sep 17 20:48:58.086258 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.085791    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 17 20:48:58.087133 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.086537    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 17 20:48:58.089408 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.089391    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-prdg7"
Sep 17 20:48:58.092292 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.091884    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 17 20:48:58.092292 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.092077    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2xkdc\""
Sep 17 20:48:58.092292 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.092202    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 17 20:48:58.096150 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.096112    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 20:48:58.115592 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.115575    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 20:48:58.115707 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.115663    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-c9kks"
Sep 17 20:48:58.123315 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.123164    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-c9kks"
Sep 17 20:48:58.167723 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.167701    2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 17 20:48:58.170178 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.170108    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-201.ec2.internal" event={"ID":"bf877a7e2526fe3fd4e830b9589fdf4b","Type":"ContainerStarted","Data":"f681fc27541516b95f08585ac3060d08c200576567495064fff98d5083575bb4"}
Sep 17 20:48:58.171458 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.171435    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-201.ec2.internal" event={"ID":"b0dbf9b7758dc823cc6b6f466c3213a7","Type":"ContainerStarted","Data":"ded08b82ef457eecdbbc71c681c47a475a12133198483e8d33622108611c9281"}
Sep 17 20:48:58.184127 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184101    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-systemd-units\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.184245 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184141    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-host-cni-netd\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.184245 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184164    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvzp6\" (UniqueName: \"kubernetes.io/projected/d17553b3-5be3-4200-adec-f293aa7a4f81-kube-api-access-bvzp6\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.184245 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184189    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-hostroot\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.184245 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184216    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szjxp\" (UniqueName: \"kubernetes.io/projected/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-kube-api-access-szjxp\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.184245 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184215    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-systemd-units\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.184245 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184239    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b1dec440-9d21-409e-a87b-7de07d399656-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zt8xk\" (UID: \"b1dec440-9d21-409e-a87b-7de07d399656\") " pod="openshift-multus/multus-additional-cni-plugins-zt8xk"
Sep 17 20:48:58.184490 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184266    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/937db66c-5b56-482e-8402-4fd3fe1d388c-agent-certs\") pod \"konnectivity-agent-4j6w6\" (UID: \"937db66c-5b56-482e-8402-4fd3fe1d388c\") " pod="kube-system/konnectivity-agent-4j6w6"
Sep 17 20:48:58.184490 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184288    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d17553b3-5be3-4200-adec-f293aa7a4f81-ovn-node-metrics-cert\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.184490 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184315    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-lib-modules\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.184490 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184351    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-host-run-multus-certs\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.184490 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184383    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgcv5\" (UniqueName: \"kubernetes.io/projected/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-kube-api-access-fgcv5\") pod \"network-metrics-daemon-jhhl5\" (UID: \"b4f3e778-0cec-45fe-be1b-a8789d63c1d7\") " pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:48:58.184490 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184406    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-var-lib-kubelet\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.184490 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184427    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-host\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.184490 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184450    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-multus-daemon-config\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.184490 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184474    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a5fe1eef-ed7f-416d-9ff9-c790010c3024-registration-dir\") pod \"aws-ebs-csi-driver-node-c5rb9\" (UID: \"a5fe1eef-ed7f-416d-9ff9-c790010c3024\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5rb9"
Sep 17 20:48:58.184863 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184496    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-host-kubelet\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.184863 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184535    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-etc-openvswitch\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.184863 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184564    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-cni-binary-copy\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.184863 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184588    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1dec440-9d21-409e-a87b-7de07d399656-cnibin\") pod \"multus-additional-cni-plugins-zt8xk\" (UID: \"b1dec440-9d21-409e-a87b-7de07d399656\") " pod="openshift-multus/multus-additional-cni-plugins-zt8xk"
Sep 17 20:48:58.184863 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184612    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/447ec589-f25d-47fb-9713-519a04467695-iptables-alerter-script\") pod \"iptables-alerter-llmv2\" (UID: \"447ec589-f25d-47fb-9713-519a04467695\") " pod="openshift-network-operator/iptables-alerter-llmv2"
Sep 17 20:48:58.184863 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184639    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-host-slash\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.184863 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184660    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6bc4ca39-a164-4ff7-9c1c-fedf495ccb5c-host\") pod \"node-ca-6h956\" (UID: \"6bc4ca39-a164-4ff7-9c1c-fedf495ccb5c\") " pod="openshift-image-registry/node-ca-6h956"
Sep 17 20:48:58.184863 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184683    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1dec440-9d21-409e-a87b-7de07d399656-cni-binary-copy\") pod \"multus-additional-cni-plugins-zt8xk\" (UID: \"b1dec440-9d21-409e-a87b-7de07d399656\") " pod="openshift-multus/multus-additional-cni-plugins-zt8xk"
Sep 17 20:48:58.184863 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184707    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxrhq\" (UniqueName: \"kubernetes.io/projected/b1dec440-9d21-409e-a87b-7de07d399656-kube-api-access-gxrhq\") pod \"multus-additional-cni-plugins-zt8xk\" (UID: \"b1dec440-9d21-409e-a87b-7de07d399656\") " pod="openshift-multus/multus-additional-cni-plugins-zt8xk"
Sep 17 20:48:58.184863 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184729    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-metrics-certs\") pod \"network-metrics-daemon-jhhl5\" (UID: \"b4f3e778-0cec-45fe-be1b-a8789d63c1d7\") " pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:48:58.184863 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184752    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-etc-systemd\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.184863 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184792    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-system-cni-dir\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.184863 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184816    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-host-var-lib-kubelet\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.184863 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184854    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-multus-conf-dir\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.184863 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184871    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1dec440-9d21-409e-a87b-7de07d399656-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zt8xk\" (UID: \"b1dec440-9d21-409e-a87b-7de07d399656\") " pod="openshift-multus/multus-additional-cni-plugins-zt8xk"
Sep 17 20:48:58.185616 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184888    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a5fe1eef-ed7f-416d-9ff9-c790010c3024-socket-dir\") pod \"aws-ebs-csi-driver-node-c5rb9\" (UID: \"a5fe1eef-ed7f-416d-9ff9-c790010c3024\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5rb9"
Sep 17 20:48:58.185616 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184903    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a5fe1eef-ed7f-416d-9ff9-c790010c3024-etc-selinux\") pod \"aws-ebs-csi-driver-node-c5rb9\" (UID: \"a5fe1eef-ed7f-416d-9ff9-c790010c3024\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5rb9"
Sep 17 20:48:58.185616 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184923    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/447ec589-f25d-47fb-9713-519a04467695-host-slash\") pod \"iptables-alerter-llmv2\" (UID: \"447ec589-f25d-47fb-9713-519a04467695\") " pod="openshift-network-operator/iptables-alerter-llmv2"
Sep 17 20:48:58.185616 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184948    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-host-run-k8s-cni-cncf-io\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.185616 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184966    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-host-cni-netd\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.185616 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184970    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a5fe1eef-ed7f-416d-9ff9-c790010c3024-sys-fs\") pod \"aws-ebs-csi-driver-node-c5rb9\" (UID: \"a5fe1eef-ed7f-416d-9ff9-c790010c3024\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5rb9"
Sep 17 20:48:58.185616 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.184994    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-host-run-multus-certs\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.185616 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.185019    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1dec440-9d21-409e-a87b-7de07d399656-cnibin\") pod \"multus-additional-cni-plugins-zt8xk\" (UID: \"b1dec440-9d21-409e-a87b-7de07d399656\") " pod="openshift-multus/multus-additional-cni-plugins-zt8xk"
Sep 17 20:48:58.185616 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.185018    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-var-lib-openvswitch\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.185616 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.185035    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a5fe1eef-ed7f-416d-9ff9-c790010c3024-registration-dir\") pod \"aws-ebs-csi-driver-node-c5rb9\" (UID: \"a5fe1eef-ed7f-416d-9ff9-c790010c3024\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5rb9"
Sep 17 20:48:58.185616 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.185057    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d17553b3-5be3-4200-adec-f293aa7a4f81-ovnkube-config\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.185616 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.185080    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-host-kubelet\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.185616 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.185084    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/082f604e-de6c-43c3-bd30-02318c0e01a4-kubelet-config\") pod \"global-pull-secret-syncer-r7jbg\" (UID: \"082f604e-de6c-43c3-bd30-02318c0e01a4\") " pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:48:58.185616 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.185112    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-etc-sysctl-conf\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.185616 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.185248    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-etc-openvswitch\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.185616 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.185310    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a5fe1eef-ed7f-416d-9ff9-c790010c3024-etc-selinux\") pod \"aws-ebs-csi-driver-node-c5rb9\" (UID: \"a5fe1eef-ed7f-416d-9ff9-c790010c3024\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5rb9"
Sep 17 20:48:58.185616 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.185358    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/447ec589-f25d-47fb-9713-519a04467695-host-slash\") pod \"iptables-alerter-llmv2\" (UID: \"447ec589-f25d-47fb-9713-519a04467695\") " pod="openshift-network-operator/iptables-alerter-llmv2"
Sep 17 20:48:58.186426 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.185393    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-host-var-lib-kubelet\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.186426 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.185595    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b1dec440-9d21-409e-a87b-7de07d399656-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zt8xk\" (UID: \"b1dec440-9d21-409e-a87b-7de07d399656\") " pod="openshift-multus/multus-additional-cni-plugins-zt8xk"
Sep 17 20:48:58.186426 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.185607    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-cni-binary-copy\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.186426 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.185598    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39f41ed8-5ecd-42b7-8d06-786ad22e316a-tmp\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.186426 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.185649    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-host-var-lib-cni-bin\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.186426 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.185676    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-run-systemd\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.186426 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.185702    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/082f604e-de6c-43c3-bd30-02318c0e01a4-original-pull-secret\") pod \"global-pull-secret-syncer-r7jbg\" (UID: \"082f604e-de6c-43c3-bd30-02318c0e01a4\") " pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:48:58.186426 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.185728    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-cnibin\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.186426 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.185752    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.186426 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.185801    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/082f604e-de6c-43c3-bd30-02318c0e01a4-dbus\") pod \"global-pull-secret-syncer-r7jbg\" (UID: \"082f604e-de6c-43c3-bd30-02318c0e01a4\") " pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:48:58.186426 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.185827    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-etc-modprobe-d\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.186426 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.185848    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-host-slash\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.186426 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.185853    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-multus-cni-dir\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.186426 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.185880    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gstzd\" (UniqueName: \"kubernetes.io/projected/447ec589-f25d-47fb-9713-519a04467695-kube-api-access-gstzd\") pod \"iptables-alerter-llmv2\" (UID: \"447ec589-f25d-47fb-9713-519a04467695\") " pod="openshift-network-operator/iptables-alerter-llmv2"
Sep 17 20:48:58.186426 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.185867    2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 17 20:48:58.186426 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.185910    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d17553b3-5be3-4200-adec-f293aa7a4f81-ovnkube-script-lib\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.186426 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.185939    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39cff161-a45c-4052-a183-e206fd6360da-tmp-dir\") pod \"node-resolver-prdg7\" (UID: \"39cff161-a45c-4052-a183-e206fd6360da\") " pod="openshift-dns/node-resolver-prdg7"
Sep 17 20:48:58.186426 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.185972    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-etc-kubernetes\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.187308 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.185998    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/39f41ed8-5ecd-42b7-8d06-786ad22e316a-etc-tuned\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.187308 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186072    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/447ec589-f25d-47fb-9713-519a04467695-iptables-alerter-script\") pod \"iptables-alerter-llmv2\" (UID: \"447ec589-f25d-47fb-9713-519a04467695\") " pod="openshift-network-operator/iptables-alerter-llmv2"
Sep 17 20:48:58.187308 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186098    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6bc4ca39-a164-4ff7-9c1c-fedf495ccb5c-serviceca\") pod \"node-ca-6h956\" (UID: \"6bc4ca39-a164-4ff7-9c1c-fedf495ccb5c\") " pod="openshift-image-registry/node-ca-6h956"
Sep 17 20:48:58.187308 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186128    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55shf\" (UniqueName: \"kubernetes.io/projected/6bc4ca39-a164-4ff7-9c1c-fedf495ccb5c-kube-api-access-55shf\") pod \"node-ca-6h956\" (UID: \"6bc4ca39-a164-4ff7-9c1c-fedf495ccb5c\") " pod="openshift-image-registry/node-ca-6h956"
Sep 17 20:48:58.187308 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:58.186148    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 20:48:58.187308 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186154    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-node-log\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.187308 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186181    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7mbv\" (UniqueName: \"kubernetes.io/projected/c6f7a65c-e785-4e19-aed2-5c2583881c06-kube-api-access-b7mbv\") pod \"network-check-target-rb4d7\" (UID: \"c6f7a65c-e785-4e19-aed2-5c2583881c06\") " pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:48:58.187308 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:58.186207    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-metrics-certs podName:b4f3e778-0cec-45fe-be1b-a8789d63c1d7 nodeName:}" failed. No retries permitted until 2026-09-17 20:48:58.686188281 +0000 UTC m=+3.031790099 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-metrics-certs") pod "network-metrics-daemon-jhhl5" (UID: "b4f3e778-0cec-45fe-be1b-a8789d63c1d7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 20:48:58.187308 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.185889    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6bc4ca39-a164-4ff7-9c1c-fedf495ccb5c-host\") pod \"node-ca-6h956\" (UID: \"6bc4ca39-a164-4ff7-9c1c-fedf495ccb5c\") " pod="openshift-image-registry/node-ca-6h956"
Sep 17 20:48:58.187308 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186236    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bqqs\" (UniqueName: \"kubernetes.io/projected/39cff161-a45c-4052-a183-e206fd6360da-kube-api-access-9bqqs\") pod \"node-resolver-prdg7\" (UID: \"39cff161-a45c-4052-a183-e206fd6360da\") " pod="openshift-dns/node-resolver-prdg7"
Sep 17 20:48:58.187308 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186262    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-etc-sysconfig\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.187308 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186288    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdfpm\" (UniqueName: \"kubernetes.io/projected/39f41ed8-5ecd-42b7-8d06-786ad22e316a-kube-api-access-xdfpm\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.187308 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186318    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/937db66c-5b56-482e-8402-4fd3fe1d388c-konnectivity-ca\") pod \"konnectivity-agent-4j6w6\" (UID: \"937db66c-5b56-482e-8402-4fd3fe1d388c\") " pod="kube-system/konnectivity-agent-4j6w6"
Sep 17 20:48:58.187308 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186344    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-run-ovn\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.187308 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186352    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-host-var-lib-cni-bin\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.187308 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186370    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-log-socket\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.187308 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186389    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-run-systemd\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.188194 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186396    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-host-cni-bin\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.188194 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186422    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/39cff161-a45c-4052-a183-e206fd6360da-hosts-file\") pod \"node-resolver-prdg7\" (UID: \"39cff161-a45c-4052-a183-e206fd6360da\") " pod="openshift-dns/node-resolver-prdg7"
Sep 17 20:48:58.188194 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186449    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-etc-sysctl-d\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.188194 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186472    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-run\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.188194 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186488    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-multus-cni-dir\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.188194 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186319    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-system-cni-dir\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.188194 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186523    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1dec440-9d21-409e-a87b-7de07d399656-system-cni-dir\") pod \"multus-additional-cni-plugins-zt8xk\" (UID: \"b1dec440-9d21-409e-a87b-7de07d399656\") " pod="openshift-multus/multus-additional-cni-plugins-zt8xk"
Sep 17 20:48:58.188194 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186538    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-run-ovn\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.188194 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186318    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1dec440-9d21-409e-a87b-7de07d399656-cni-binary-copy\") pod \"multus-additional-cni-plugins-zt8xk\" (UID: \"b1dec440-9d21-409e-a87b-7de07d399656\") " pod="openshift-multus/multus-additional-cni-plugins-zt8xk"
Sep 17 20:48:58.188194 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186561    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a5fe1eef-ed7f-416d-9ff9-c790010c3024-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c5rb9\" (UID: \"a5fe1eef-ed7f-416d-9ff9-c790010c3024\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5rb9"
Sep 17 20:48:58.188194 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186585    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-log-socket\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.188194 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186589    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-run-openvswitch\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.188194 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186621    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-host-cni-bin\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.188194 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186633    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-run-openvswitch\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.188194 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186647    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d17553b3-5be3-4200-adec-f293aa7a4f81-env-overrides\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.188194 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186670    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1dec440-9d21-409e-a87b-7de07d399656-system-cni-dir\") pod \"multus-additional-cni-plugins-zt8xk\" (UID: \"b1dec440-9d21-409e-a87b-7de07d399656\") " pod="openshift-multus/multus-additional-cni-plugins-zt8xk"
Sep 17 20:48:58.188194 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186674    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-sys\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.189022 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186702    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-host-run-netns\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.189022 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186731    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-host-var-lib-cni-multus\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.189022 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186743    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a5fe1eef-ed7f-416d-9ff9-c790010c3024-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c5rb9\" (UID: \"a5fe1eef-ed7f-416d-9ff9-c790010c3024\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5rb9"
Sep 17 20:48:58.189022 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186774    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-host-run-ovn-kubernetes\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.189022 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186812    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-host-run-ovn-kubernetes\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.189022 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186835    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-multus-socket-dir-parent\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.189022 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186865    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4wzl\" (UniqueName: \"kubernetes.io/projected/a5fe1eef-ed7f-416d-9ff9-c790010c3024-kube-api-access-v4wzl\") pod \"aws-ebs-csi-driver-node-c5rb9\" (UID: \"a5fe1eef-ed7f-416d-9ff9-c790010c3024\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5rb9"
Sep 17 20:48:58.189022 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186894    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-host-run-netns\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.189022 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186922    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-etc-kubernetes\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.189022 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186949    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b1dec440-9d21-409e-a87b-7de07d399656-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zt8xk\" (UID: \"b1dec440-9d21-409e-a87b-7de07d399656\") " pod="openshift-multus/multus-additional-cni-plugins-zt8xk"
Sep 17 20:48:58.189022 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.186977    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a5fe1eef-ed7f-416d-9ff9-c790010c3024-device-dir\") pod \"aws-ebs-csi-driver-node-c5rb9\" (UID: \"a5fe1eef-ed7f-416d-9ff9-c790010c3024\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5rb9"
Sep 17 20:48:58.189022 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.187012    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/937db66c-5b56-482e-8402-4fd3fe1d388c-konnectivity-ca\") pod \"konnectivity-agent-4j6w6\" (UID: \"937db66c-5b56-482e-8402-4fd3fe1d388c\") " pod="kube-system/konnectivity-agent-4j6w6"
Sep 17 20:48:58.189022 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.187053    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d17553b3-5be3-4200-adec-f293aa7a4f81-ovnkube-script-lib\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.189022 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.187066    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-cnibin\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.189022 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.187098    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a5fe1eef-ed7f-416d-9ff9-c790010c3024-device-dir\") pod \"aws-ebs-csi-driver-node-c5rb9\" (UID: \"a5fe1eef-ed7f-416d-9ff9-c790010c3024\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5rb9"
Sep 17 20:48:58.189022 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.187133    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/082f604e-de6c-43c3-bd30-02318c0e01a4-dbus\") pod \"global-pull-secret-syncer-r7jbg\" (UID: \"082f604e-de6c-43c3-bd30-02318c0e01a4\") " pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:48:58.189022 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.187133    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.189857 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.187160    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d17553b3-5be3-4200-adec-f293aa7a4f81-env-overrides\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.189857 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.187177    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-multus-socket-dir-parent\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.189857 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.187183    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a5fe1eef-ed7f-416d-9ff9-c790010c3024-sys-fs\") pod \"aws-ebs-csi-driver-node-c5rb9\" (UID: \"a5fe1eef-ed7f-416d-9ff9-c790010c3024\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5rb9"
Sep 17 20:48:58.189857 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.187222    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-host-var-lib-cni-multus\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.189857 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.187222    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-multus-conf-dir\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.189857 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.187225    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-etc-kubernetes\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.189857 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.187249    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-hostroot\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.189857 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:58.187307    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 20:48:58.189857 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:58.187351    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/082f604e-de6c-43c3-bd30-02318c0e01a4-original-pull-secret podName:082f604e-de6c-43c3-bd30-02318c0e01a4 nodeName:}" failed. No retries permitted until 2026-09-17 20:48:58.687337098 +0000 UTC m=+3.032938910 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/082f604e-de6c-43c3-bd30-02318c0e01a4-original-pull-secret") pod "global-pull-secret-syncer-r7jbg" (UID: "082f604e-de6c-43c3-bd30-02318c0e01a4") : object "kube-system"/"original-pull-secret" not registered
Sep 17 20:48:58.189857 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.187396    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-host-run-netns\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.189857 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.187407    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-host-run-k8s-cni-cncf-io\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.189857 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.187429    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6bc4ca39-a164-4ff7-9c1c-fedf495ccb5c-serviceca\") pod \"node-ca-6h956\" (UID: \"6bc4ca39-a164-4ff7-9c1c-fedf495ccb5c\") " pod="openshift-image-registry/node-ca-6h956"
Sep 17 20:48:58.189857 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.187434    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-var-lib-openvswitch\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.189857 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.187524    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1dec440-9d21-409e-a87b-7de07d399656-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zt8xk\" (UID: \"b1dec440-9d21-409e-a87b-7de07d399656\") " pod="openshift-multus/multus-additional-cni-plugins-zt8xk"
Sep 17 20:48:58.189857 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.187561    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-node-log\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.189857 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.187606    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/082f604e-de6c-43c3-bd30-02318c0e01a4-kubelet-config\") pod \"global-pull-secret-syncer-r7jbg\" (UID: \"082f604e-de6c-43c3-bd30-02318c0e01a4\") " pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:48:58.189857 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.187659    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b1dec440-9d21-409e-a87b-7de07d399656-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zt8xk\" (UID: \"b1dec440-9d21-409e-a87b-7de07d399656\") " pod="openshift-multus/multus-additional-cni-plugins-zt8xk"
Sep 17 20:48:58.193153 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.187689    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a5fe1eef-ed7f-416d-9ff9-c790010c3024-socket-dir\") pod \"aws-ebs-csi-driver-node-c5rb9\" (UID: \"a5fe1eef-ed7f-416d-9ff9-c790010c3024\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5rb9"
Sep 17 20:48:58.193153 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.187730    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d17553b3-5be3-4200-adec-f293aa7a4f81-host-run-netns\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.193153 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.187962    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-multus-daemon-config\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.193153 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.188001    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d17553b3-5be3-4200-adec-f293aa7a4f81-ovnkube-config\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.193153 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.190772    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d17553b3-5be3-4200-adec-f293aa7a4f81-ovn-node-metrics-cert\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.193153 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.191110    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/937db66c-5b56-482e-8402-4fd3fe1d388c-agent-certs\") pod \"konnectivity-agent-4j6w6\" (UID: \"937db66c-5b56-482e-8402-4fd3fe1d388c\") " pod="kube-system/konnectivity-agent-4j6w6"
Sep 17 20:48:58.199154 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.199127    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvzp6\" (UniqueName: \"kubernetes.io/projected/d17553b3-5be3-4200-adec-f293aa7a4f81-kube-api-access-bvzp6\") pod \"ovnkube-node-gcr8f\" (UID: \"d17553b3-5be3-4200-adec-f293aa7a4f81\") " pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.199273 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.199253    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgcv5\" (UniqueName: \"kubernetes.io/projected/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-kube-api-access-fgcv5\") pod \"network-metrics-daemon-jhhl5\" (UID: \"b4f3e778-0cec-45fe-be1b-a8789d63c1d7\") " pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:48:58.199273 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:58.199269    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 20:48:58.199389 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:58.199295    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 20:48:58.199389 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:58.199309    2567 projected.go:196] Error preparing data for projected volume kube-api-access-b7mbv for pod openshift-network-diagnostics/network-check-target-rb4d7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 20:48:58.199389 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:58.199374    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c6f7a65c-e785-4e19-aed2-5c2583881c06-kube-api-access-b7mbv podName:c6f7a65c-e785-4e19-aed2-5c2583881c06 nodeName:}" failed. No retries permitted until 2026-09-17 20:48:58.699359283 +0000 UTC m=+3.044961088 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-b7mbv" (UniqueName: "kubernetes.io/projected/c6f7a65c-e785-4e19-aed2-5c2583881c06-kube-api-access-b7mbv") pod "network-check-target-rb4d7" (UID: "c6f7a65c-e785-4e19-aed2-5c2583881c06") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 20:48:58.201798 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.201708    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4wzl\" (UniqueName: \"kubernetes.io/projected/a5fe1eef-ed7f-416d-9ff9-c790010c3024-kube-api-access-v4wzl\") pod \"aws-ebs-csi-driver-node-c5rb9\" (UID: \"a5fe1eef-ed7f-416d-9ff9-c790010c3024\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5rb9"
Sep 17 20:48:58.201798 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.201713    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55shf\" (UniqueName: \"kubernetes.io/projected/6bc4ca39-a164-4ff7-9c1c-fedf495ccb5c-kube-api-access-55shf\") pod \"node-ca-6h956\" (UID: \"6bc4ca39-a164-4ff7-9c1c-fedf495ccb5c\") " pod="openshift-image-registry/node-ca-6h956"
Sep 17 20:48:58.201798 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.201713    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szjxp\" (UniqueName: \"kubernetes.io/projected/f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b-kube-api-access-szjxp\") pod \"multus-4bpjd\" (UID: \"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b\") " pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.203122 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.203102    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxrhq\" (UniqueName: \"kubernetes.io/projected/b1dec440-9d21-409e-a87b-7de07d399656-kube-api-access-gxrhq\") pod \"multus-additional-cni-plugins-zt8xk\" (UID: \"b1dec440-9d21-409e-a87b-7de07d399656\") " pod="openshift-multus/multus-additional-cni-plugins-zt8xk"
Sep 17 20:48:58.203646 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.203604    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gstzd\" (UniqueName: \"kubernetes.io/projected/447ec589-f25d-47fb-9713-519a04467695-kube-api-access-gstzd\") pod \"iptables-alerter-llmv2\" (UID: \"447ec589-f25d-47fb-9713-519a04467695\") " pod="openshift-network-operator/iptables-alerter-llmv2"
Sep 17 20:48:58.288123 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.288090    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-etc-systemd\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.288284 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.288139    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-etc-sysctl-conf\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.288284 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.288163    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39f41ed8-5ecd-42b7-8d06-786ad22e316a-tmp\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.288284 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.288200    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-etc-modprobe-d\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.288284 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.288226    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39cff161-a45c-4052-a183-e206fd6360da-tmp-dir\") pod \"node-resolver-prdg7\" (UID: \"39cff161-a45c-4052-a183-e206fd6360da\") " pod="openshift-dns/node-resolver-prdg7"
Sep 17 20:48:58.288284 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.288243    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-etc-systemd\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.288284 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.288248    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-etc-kubernetes\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.288535 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.288301    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-etc-kubernetes\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.288535 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.288302    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/39f41ed8-5ecd-42b7-8d06-786ad22e316a-etc-tuned\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.288535 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.288353    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bqqs\" (UniqueName: \"kubernetes.io/projected/39cff161-a45c-4052-a183-e206fd6360da-kube-api-access-9bqqs\") pod \"node-resolver-prdg7\" (UID: \"39cff161-a45c-4052-a183-e206fd6360da\") " pod="openshift-dns/node-resolver-prdg7"
Sep 17 20:48:58.288535 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.288376    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-etc-sysconfig\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.288535 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.288398    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdfpm\" (UniqueName: \"kubernetes.io/projected/39f41ed8-5ecd-42b7-8d06-786ad22e316a-kube-api-access-xdfpm\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.288535 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.288424    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/39cff161-a45c-4052-a183-e206fd6360da-hosts-file\") pod \"node-resolver-prdg7\" (UID: \"39cff161-a45c-4052-a183-e206fd6360da\") " pod="openshift-dns/node-resolver-prdg7"
Sep 17 20:48:58.288535 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.288443    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-etc-sysctl-d\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.288535 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.288461    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-run\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.288535 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.288482    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-sys\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.288535 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.288502    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-etc-modprobe-d\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.288535 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.288516    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-lib-modules\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.288535 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.288526    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-etc-sysctl-conf\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.289056 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.288549    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-var-lib-kubelet\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.289056 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.288582    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-host\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.289056 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.288625    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-lib-modules\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.289056 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.288655    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-host\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.289056 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.288656    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-var-lib-kubelet\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.289056 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.288679    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-etc-sysconfig\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.289056 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.288713    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-sys\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.289056 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.288817    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/39cff161-a45c-4052-a183-e206fd6360da-hosts-file\") pod \"node-resolver-prdg7\" (UID: \"39cff161-a45c-4052-a183-e206fd6360da\") " pod="openshift-dns/node-resolver-prdg7"
Sep 17 20:48:58.289056 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.288879    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-run\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.289056 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.288887    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/39f41ed8-5ecd-42b7-8d06-786ad22e316a-etc-sysctl-d\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.289056 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.288880    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39cff161-a45c-4052-a183-e206fd6360da-tmp-dir\") pod \"node-resolver-prdg7\" (UID: \"39cff161-a45c-4052-a183-e206fd6360da\") " pod="openshift-dns/node-resolver-prdg7"
Sep 17 20:48:58.291110 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.291087    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/39f41ed8-5ecd-42b7-8d06-786ad22e316a-etc-tuned\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.291212 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.291136    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39f41ed8-5ecd-42b7-8d06-786ad22e316a-tmp\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.297505 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.297451    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdfpm\" (UniqueName: \"kubernetes.io/projected/39f41ed8-5ecd-42b7-8d06-786ad22e316a-kube-api-access-xdfpm\") pod \"tuned-s9k9h\" (UID: \"39f41ed8-5ecd-42b7-8d06-786ad22e316a\") " pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.297505 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.297451    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bqqs\" (UniqueName: \"kubernetes.io/projected/39cff161-a45c-4052-a183-e206fd6360da-kube-api-access-9bqqs\") pod \"node-resolver-prdg7\" (UID: \"39cff161-a45c-4052-a183-e206fd6360da\") " pod="openshift-dns/node-resolver-prdg7"
Sep 17 20:48:58.384731 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.384691    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4j6w6"
Sep 17 20:48:58.393492 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.393437    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4bpjd"
Sep 17 20:48:58.396597 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:58.396569    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod937db66c_5b56_482e_8402_4fd3fe1d388c.slice/crio-199f9e97c6567416e4e79a571422134a3810130357010e052a1408527e5bd468 WatchSource:0}: Error finding container 199f9e97c6567416e4e79a571422134a3810130357010e052a1408527e5bd468: Status 404 returned error can't find the container with id 199f9e97c6567416e4e79a571422134a3810130357010e052a1408527e5bd468
Sep 17 20:48:58.403088 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.403050    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6h956"
Sep 17 20:48:58.404856 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:58.404823    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0454a2d_7d6a_4e88_bfdc_5f4dd44e754b.slice/crio-b3d0250b7b28c90a8f6776346cbbc9692539d6fbb7a0547fc71a1d8f51dda494 WatchSource:0}: Error finding container b3d0250b7b28c90a8f6776346cbbc9692539d6fbb7a0547fc71a1d8f51dda494: Status 404 returned error can't find the container with id b3d0250b7b28c90a8f6776346cbbc9692539d6fbb7a0547fc71a1d8f51dda494
Sep 17 20:48:58.407920 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.407721    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:48:58.414565 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:58.414539    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bc4ca39_a164_4ff7_9c1c_fedf495ccb5c.slice/crio-3521637d3d16dce629d46e6b9778b82b4120858b527648852c509e4f39e27a88 WatchSource:0}: Error finding container 3521637d3d16dce629d46e6b9778b82b4120858b527648852c509e4f39e27a88: Status 404 returned error can't find the container with id 3521637d3d16dce629d46e6b9778b82b4120858b527648852c509e4f39e27a88
Sep 17 20:48:58.415485 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.415468    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zt8xk"
Sep 17 20:48:58.417484 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:58.417460    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd17553b3_5be3_4200_adec_f293aa7a4f81.slice/crio-82e37b1090e79fb41f9f1c16b7881e3abb52311505331bf790ec573c5e26e689 WatchSource:0}: Error finding container 82e37b1090e79fb41f9f1c16b7881e3abb52311505331bf790ec573c5e26e689: Status 404 returned error can't find the container with id 82e37b1090e79fb41f9f1c16b7881e3abb52311505331bf790ec573c5e26e689
Sep 17 20:48:58.424219 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.424199    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5rb9"
Sep 17 20:48:58.433728 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.433689    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-llmv2"
Sep 17 20:48:58.448408 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.448389    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-s9k9h"
Sep 17 20:48:58.454504 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.454486    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-prdg7"
Sep 17 20:48:58.690420 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.690343    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-metrics-certs\") pod \"network-metrics-daemon-jhhl5\" (UID: \"b4f3e778-0cec-45fe-be1b-a8789d63c1d7\") " pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:48:58.690420 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.690386    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/082f604e-de6c-43c3-bd30-02318c0e01a4-original-pull-secret\") pod \"global-pull-secret-syncer-r7jbg\" (UID: \"082f604e-de6c-43c3-bd30-02318c0e01a4\") " pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:48:58.690640 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:58.690525    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 20:48:58.690640 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:58.690600    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-metrics-certs podName:b4f3e778-0cec-45fe-be1b-a8789d63c1d7 nodeName:}" failed. No retries permitted until 2026-09-17 20:48:59.690581045 +0000 UTC m=+4.036182862 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-metrics-certs") pod "network-metrics-daemon-jhhl5" (UID: "b4f3e778-0cec-45fe-be1b-a8789d63c1d7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 20:48:58.690640 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:58.690616    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 20:48:58.690825 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:58.690674    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/082f604e-de6c-43c3-bd30-02318c0e01a4-original-pull-secret podName:082f604e-de6c-43c3-bd30-02318c0e01a4 nodeName:}" failed. No retries permitted until 2026-09-17 20:48:59.69065802 +0000 UTC m=+4.036259835 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/082f604e-de6c-43c3-bd30-02318c0e01a4-original-pull-secret") pod "global-pull-secret-syncer-r7jbg" (UID: "082f604e-de6c-43c3-bd30-02318c0e01a4") : object "kube-system"/"original-pull-secret" not registered
Sep 17 20:48:58.791198 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:58.791167    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7mbv\" (UniqueName: \"kubernetes.io/projected/c6f7a65c-e785-4e19-aed2-5c2583881c06-kube-api-access-b7mbv\") pod \"network-check-target-rb4d7\" (UID: \"c6f7a65c-e785-4e19-aed2-5c2583881c06\") " pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:48:58.791348 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:58.791319    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 20:48:58.791348 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:58.791339    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 20:48:58.791348 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:58.791350    2567 projected.go:196] Error preparing data for projected volume kube-api-access-b7mbv for pod openshift-network-diagnostics/network-check-target-rb4d7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 20:48:58.791478 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:58.791400    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c6f7a65c-e785-4e19-aed2-5c2583881c06-kube-api-access-b7mbv podName:c6f7a65c-e785-4e19-aed2-5c2583881c06 nodeName:}" failed. No retries permitted until 2026-09-17 20:48:59.791385262 +0000 UTC m=+4.136987076 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-b7mbv" (UniqueName: "kubernetes.io/projected/c6f7a65c-e785-4e19-aed2-5c2583881c06-kube-api-access-b7mbv") pod "network-check-target-rb4d7" (UID: "c6f7a65c-e785-4e19-aed2-5c2583881c06") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 20:48:59.126591 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:59.126508    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 20:43:58 +0000 UTC" deadline="2028-05-01 23:34:25.536376388 +0000 UTC"
Sep 17 20:48:59.126591 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:59.126541    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14210h45m26.409840106s"
Sep 17 20:48:59.174393 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:59.174357    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f" event={"ID":"d17553b3-5be3-4200-adec-f293aa7a4f81","Type":"ContainerStarted","Data":"82e37b1090e79fb41f9f1c16b7881e3abb52311505331bf790ec573c5e26e689"}
Sep 17 20:48:59.175637 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:59.175612    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6h956" event={"ID":"6bc4ca39-a164-4ff7-9c1c-fedf495ccb5c","Type":"ContainerStarted","Data":"3521637d3d16dce629d46e6b9778b82b4120858b527648852c509e4f39e27a88"}
Sep 17 20:48:59.176783 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:59.176744    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4bpjd" event={"ID":"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b","Type":"ContainerStarted","Data":"b3d0250b7b28c90a8f6776346cbbc9692539d6fbb7a0547fc71a1d8f51dda494"}
Sep 17 20:48:59.177820 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:59.177798    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4j6w6" event={"ID":"937db66c-5b56-482e-8402-4fd3fe1d388c","Type":"ContainerStarted","Data":"199f9e97c6567416e4e79a571422134a3810130357010e052a1408527e5bd468"}
Sep 17 20:48:59.431230 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:59.431204    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39cff161_a45c_4052_a183_e206fd6360da.slice/crio-1562093da6090cb08a93b88298db571e811556802b210ab1189731aefb147604 WatchSource:0}: Error finding container 1562093da6090cb08a93b88298db571e811556802b210ab1189731aefb147604: Status 404 returned error can't find the container with id 1562093da6090cb08a93b88298db571e811556802b210ab1189731aefb147604
Sep 17 20:48:59.432769 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:59.432725    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod447ec589_f25d_47fb_9713_519a04467695.slice/crio-15a19f846800ff82522b4bf25c623976b554cce2a93a30c2e5e83aacfc4e3d32 WatchSource:0}: Error finding container 15a19f846800ff82522b4bf25c623976b554cce2a93a30c2e5e83aacfc4e3d32: Status 404 returned error can't find the container with id 15a19f846800ff82522b4bf25c623976b554cce2a93a30c2e5e83aacfc4e3d32
Sep 17 20:48:59.433905 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:59.433873    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1dec440_9d21_409e_a87b_7de07d399656.slice/crio-6edcd65db6c5c9d32de77547651e4fb7261c4c477b773496b89ae6440ccc633a WatchSource:0}: Error finding container 6edcd65db6c5c9d32de77547651e4fb7261c4c477b773496b89ae6440ccc633a: Status 404 returned error can't find the container with id 6edcd65db6c5c9d32de77547651e4fb7261c4c477b773496b89ae6440ccc633a
Sep 17 20:48:59.434989 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:59.434963    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5fe1eef_ed7f_416d_9ff9_c790010c3024.slice/crio-8d67c169c1e03b8debcfc080e0b015fc9d4e2f2adc7fcf4e0847b092e0034efe WatchSource:0}: Error finding container 8d67c169c1e03b8debcfc080e0b015fc9d4e2f2adc7fcf4e0847b092e0034efe: Status 404 returned error can't find the container with id 8d67c169c1e03b8debcfc080e0b015fc9d4e2f2adc7fcf4e0847b092e0034efe
Sep 17 20:48:59.455204 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:48:59.455159    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39f41ed8_5ecd_42b7_8d06_786ad22e316a.slice/crio-0104912d6f476c2fa2b7034d81e5739e5edfe62dd3446750aa94d31c495eb100 WatchSource:0}: Error finding container 0104912d6f476c2fa2b7034d81e5739e5edfe62dd3446750aa94d31c495eb100: Status 404 returned error can't find the container with id 0104912d6f476c2fa2b7034d81e5739e5edfe62dd3446750aa94d31c495eb100
Sep 17 20:48:59.697892 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:59.697782    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-metrics-certs\") pod \"network-metrics-daemon-jhhl5\" (UID: \"b4f3e778-0cec-45fe-be1b-a8789d63c1d7\") " pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:48:59.697892 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:59.697841    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/082f604e-de6c-43c3-bd30-02318c0e01a4-original-pull-secret\") pod \"global-pull-secret-syncer-r7jbg\" (UID: \"082f604e-de6c-43c3-bd30-02318c0e01a4\") " pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:48:59.698159 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:59.697960    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 20:48:59.698159 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:59.697974    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 20:48:59.698159 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:59.698028    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/082f604e-de6c-43c3-bd30-02318c0e01a4-original-pull-secret podName:082f604e-de6c-43c3-bd30-02318c0e01a4 nodeName:}" failed. No retries permitted until 2026-09-17 20:49:01.698011375 +0000 UTC m=+6.043613192 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/082f604e-de6c-43c3-bd30-02318c0e01a4-original-pull-secret") pod "global-pull-secret-syncer-r7jbg" (UID: "082f604e-de6c-43c3-bd30-02318c0e01a4") : object "kube-system"/"original-pull-secret" not registered
Sep 17 20:48:59.698159 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:59.698047    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-metrics-certs podName:b4f3e778-0cec-45fe-be1b-a8789d63c1d7 nodeName:}" failed. No retries permitted until 2026-09-17 20:49:01.698037068 +0000 UTC m=+6.043638872 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-metrics-certs") pod "network-metrics-daemon-jhhl5" (UID: "b4f3e778-0cec-45fe-be1b-a8789d63c1d7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 20:48:59.799318 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:48:59.799282    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7mbv\" (UniqueName: \"kubernetes.io/projected/c6f7a65c-e785-4e19-aed2-5c2583881c06-kube-api-access-b7mbv\") pod \"network-check-target-rb4d7\" (UID: \"c6f7a65c-e785-4e19-aed2-5c2583881c06\") " pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:48:59.799499 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:59.799451    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 20:48:59.799499 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:59.799473    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 20:48:59.799499 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:59.799486    2567 projected.go:196] Error preparing data for projected volume kube-api-access-b7mbv for pod openshift-network-diagnostics/network-check-target-rb4d7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 20:48:59.799661 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:48:59.799539    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c6f7a65c-e785-4e19-aed2-5c2583881c06-kube-api-access-b7mbv podName:c6f7a65c-e785-4e19-aed2-5c2583881c06 nodeName:}" failed. No retries permitted until 2026-09-17 20:49:01.799521846 +0000 UTC m=+6.145123664 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-b7mbv" (UniqueName: "kubernetes.io/projected/c6f7a65c-e785-4e19-aed2-5c2583881c06-kube-api-access-b7mbv") pod "network-check-target-rb4d7" (UID: "c6f7a65c-e785-4e19-aed2-5c2583881c06") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 20:49:00.127619 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:00.127549    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 20:43:58 +0000 UTC" deadline="2028-05-27 01:08:50.196700646 +0000 UTC"
Sep 17 20:49:00.127619 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:00.127608    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14812h19m50.069096519s"
Sep 17 20:49:00.129522 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:00.129293    2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 20:49:00.167944 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:00.167113    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:49:00.167944 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:00.167245    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r7jbg" podUID="082f604e-de6c-43c3-bd30-02318c0e01a4"
Sep 17 20:49:00.167944 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:00.167644    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:49:00.167944 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:00.167733    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rb4d7" podUID="c6f7a65c-e785-4e19-aed2-5c2583881c06"
Sep 17 20:49:00.167944 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:00.167812    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:49:00.167944 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:00.167893    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhhl5" podUID="b4f3e778-0cec-45fe-be1b-a8789d63c1d7"
Sep 17 20:49:00.186022 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:00.185979    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-201.ec2.internal" event={"ID":"bf877a7e2526fe3fd4e830b9589fdf4b","Type":"ContainerStarted","Data":"a1c802937d647bc2875de78062704f9d93f0e21828c8521a39a0ffa61c1672c7"}
Sep 17 20:49:00.195290 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:00.195254    2567 generic.go:356] "Generic (PLEG): container finished" podID="b0dbf9b7758dc823cc6b6f466c3213a7" containerID="ef20eb2e564b201b6db074e536c35c75547c430ebdaf7a8820971412005c6a8c" exitCode=0
Sep 17 20:49:00.195470 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:00.195347    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-201.ec2.internal" event={"ID":"b0dbf9b7758dc823cc6b6f466c3213a7","Type":"ContainerDied","Data":"ef20eb2e564b201b6db074e536c35c75547c430ebdaf7a8820971412005c6a8c"}
Sep 17 20:49:00.203145 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:00.203105    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zt8xk" event={"ID":"b1dec440-9d21-409e-a87b-7de07d399656","Type":"ContainerStarted","Data":"6edcd65db6c5c9d32de77547651e4fb7261c4c477b773496b89ae6440ccc633a"}
Sep 17 20:49:00.210955 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:00.210884    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-llmv2" event={"ID":"447ec589-f25d-47fb-9713-519a04467695","Type":"ContainerStarted","Data":"15a19f846800ff82522b4bf25c623976b554cce2a93a30c2e5e83aacfc4e3d32"}
Sep 17 20:49:00.215079 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:00.215020    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-s9k9h" event={"ID":"39f41ed8-5ecd-42b7-8d06-786ad22e316a","Type":"ContainerStarted","Data":"0104912d6f476c2fa2b7034d81e5739e5edfe62dd3446750aa94d31c495eb100"}
Sep 17 20:49:00.220083 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:00.220031    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-201.ec2.internal" podStartSLOduration=3.220011091 podStartE2EDuration="3.220011091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 20:48:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:49:00.205056755 +0000 UTC m=+4.550658582" watchObservedRunningTime="2026-09-17 20:49:00.220011091 +0000 UTC m=+4.565612911"
Sep 17 20:49:00.222135 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:00.222105    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5rb9" event={"ID":"a5fe1eef-ed7f-416d-9ff9-c790010c3024","Type":"ContainerStarted","Data":"8d67c169c1e03b8debcfc080e0b015fc9d4e2f2adc7fcf4e0847b092e0034efe"}
Sep 17 20:49:00.224770 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:00.224731    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-prdg7" event={"ID":"39cff161-a45c-4052-a183-e206fd6360da","Type":"ContainerStarted","Data":"1562093da6090cb08a93b88298db571e811556802b210ab1189731aefb147604"}
Sep 17 20:49:01.239234 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:01.238927    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-201.ec2.internal" event={"ID":"b0dbf9b7758dc823cc6b6f466c3213a7","Type":"ContainerStarted","Data":"a3704eb05fbb9d5db7197ab4cb6be75d43fd160ad202f399197e8eb46a22696f"}
Sep 17 20:49:01.719381 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:01.719293    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/082f604e-de6c-43c3-bd30-02318c0e01a4-original-pull-secret\") pod \"global-pull-secret-syncer-r7jbg\" (UID: \"082f604e-de6c-43c3-bd30-02318c0e01a4\") " pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:49:01.719537 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:01.719400    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-metrics-certs\") pod \"network-metrics-daemon-jhhl5\" (UID: \"b4f3e778-0cec-45fe-be1b-a8789d63c1d7\") " pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:49:01.719537 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:01.719516    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 20:49:01.719641 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:01.719589    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-metrics-certs podName:b4f3e778-0cec-45fe-be1b-a8789d63c1d7 nodeName:}" failed. No retries permitted until 2026-09-17 20:49:05.719569133 +0000 UTC m=+10.065170939 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-metrics-certs") pod "network-metrics-daemon-jhhl5" (UID: "b4f3e778-0cec-45fe-be1b-a8789d63c1d7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 20:49:01.719963 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:01.719516    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 20:49:01.719963 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:01.719963    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/082f604e-de6c-43c3-bd30-02318c0e01a4-original-pull-secret podName:082f604e-de6c-43c3-bd30-02318c0e01a4 nodeName:}" failed. No retries permitted until 2026-09-17 20:49:05.719947984 +0000 UTC m=+10.065549792 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/082f604e-de6c-43c3-bd30-02318c0e01a4-original-pull-secret") pod "global-pull-secret-syncer-r7jbg" (UID: "082f604e-de6c-43c3-bd30-02318c0e01a4") : object "kube-system"/"original-pull-secret" not registered
Sep 17 20:49:01.820224 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:01.820127    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7mbv\" (UniqueName: \"kubernetes.io/projected/c6f7a65c-e785-4e19-aed2-5c2583881c06-kube-api-access-b7mbv\") pod \"network-check-target-rb4d7\" (UID: \"c6f7a65c-e785-4e19-aed2-5c2583881c06\") " pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:49:01.820388 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:01.820287    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 20:49:01.820388 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:01.820305    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 20:49:01.820388 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:01.820318    2567 projected.go:196] Error preparing data for projected volume kube-api-access-b7mbv for pod openshift-network-diagnostics/network-check-target-rb4d7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 20:49:01.820388 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:01.820377    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c6f7a65c-e785-4e19-aed2-5c2583881c06-kube-api-access-b7mbv podName:c6f7a65c-e785-4e19-aed2-5c2583881c06 nodeName:}" failed. No retries permitted until 2026-09-17 20:49:05.820357981 +0000 UTC m=+10.165959799 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-b7mbv" (UniqueName: "kubernetes.io/projected/c6f7a65c-e785-4e19-aed2-5c2583881c06-kube-api-access-b7mbv") pod "network-check-target-rb4d7" (UID: "c6f7a65c-e785-4e19-aed2-5c2583881c06") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 20:49:02.167028 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:02.166066    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:49:02.167028 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:02.166205    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhhl5" podUID="b4f3e778-0cec-45fe-be1b-a8789d63c1d7"
Sep 17 20:49:02.167028 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:02.166618    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:49:02.167028 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:02.166714    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r7jbg" podUID="082f604e-de6c-43c3-bd30-02318c0e01a4"
Sep 17 20:49:02.167028 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:02.166796    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:49:02.167028 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:02.166862    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rb4d7" podUID="c6f7a65c-e785-4e19-aed2-5c2583881c06"
Sep 17 20:49:04.164840 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:04.164811    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:49:04.165280 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:04.164846    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:49:04.165280 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:04.164811    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:49:04.165280 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:04.164943    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rb4d7" podUID="c6f7a65c-e785-4e19-aed2-5c2583881c06"
Sep 17 20:49:04.165280 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:04.165085    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r7jbg" podUID="082f604e-de6c-43c3-bd30-02318c0e01a4"
Sep 17 20:49:04.165280 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:04.165187    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhhl5" podUID="b4f3e778-0cec-45fe-be1b-a8789d63c1d7"
Sep 17 20:49:05.753461 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:05.752711    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/082f604e-de6c-43c3-bd30-02318c0e01a4-original-pull-secret\") pod \"global-pull-secret-syncer-r7jbg\" (UID: \"082f604e-de6c-43c3-bd30-02318c0e01a4\") " pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:49:05.753461 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:05.752822    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-metrics-certs\") pod \"network-metrics-daemon-jhhl5\" (UID: \"b4f3e778-0cec-45fe-be1b-a8789d63c1d7\") " pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:49:05.753461 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:05.752954    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 20:49:05.753461 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:05.753020    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-metrics-certs podName:b4f3e778-0cec-45fe-be1b-a8789d63c1d7 nodeName:}" failed. No retries permitted until 2026-09-17 20:49:13.752999652 +0000 UTC m=+18.098601478 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-metrics-certs") pod "network-metrics-daemon-jhhl5" (UID: "b4f3e778-0cec-45fe-be1b-a8789d63c1d7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 20:49:05.753461 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:05.753082    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 20:49:05.753461 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:05.753115    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/082f604e-de6c-43c3-bd30-02318c0e01a4-original-pull-secret podName:082f604e-de6c-43c3-bd30-02318c0e01a4 nodeName:}" failed. No retries permitted until 2026-09-17 20:49:13.753104157 +0000 UTC m=+18.098705970 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/082f604e-de6c-43c3-bd30-02318c0e01a4-original-pull-secret") pod "global-pull-secret-syncer-r7jbg" (UID: "082f604e-de6c-43c3-bd30-02318c0e01a4") : object "kube-system"/"original-pull-secret" not registered
Sep 17 20:49:05.853628 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:05.853591    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7mbv\" (UniqueName: \"kubernetes.io/projected/c6f7a65c-e785-4e19-aed2-5c2583881c06-kube-api-access-b7mbv\") pod \"network-check-target-rb4d7\" (UID: \"c6f7a65c-e785-4e19-aed2-5c2583881c06\") " pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:49:05.853806 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:05.853790    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 20:49:05.853864 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:05.853813    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 20:49:05.853864 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:05.853827    2567 projected.go:196] Error preparing data for projected volume kube-api-access-b7mbv for pod openshift-network-diagnostics/network-check-target-rb4d7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 20:49:05.853967 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:05.853889    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c6f7a65c-e785-4e19-aed2-5c2583881c06-kube-api-access-b7mbv podName:c6f7a65c-e785-4e19-aed2-5c2583881c06 nodeName:}" failed. No retries permitted until 2026-09-17 20:49:13.853873576 +0000 UTC m=+18.199475384 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-b7mbv" (UniqueName: "kubernetes.io/projected/c6f7a65c-e785-4e19-aed2-5c2583881c06-kube-api-access-b7mbv") pod "network-check-target-rb4d7" (UID: "c6f7a65c-e785-4e19-aed2-5c2583881c06") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 20:49:06.165037 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:06.164924    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:49:06.165037 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:06.165011    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:49:06.165250 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:06.165055    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:49:06.165250 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:06.165137    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhhl5" podUID="b4f3e778-0cec-45fe-be1b-a8789d63c1d7"
Sep 17 20:49:06.165503 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:06.165425    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rb4d7" podUID="c6f7a65c-e785-4e19-aed2-5c2583881c06"
Sep 17 20:49:06.165503 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:06.165468    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r7jbg" podUID="082f604e-de6c-43c3-bd30-02318c0e01a4"
Sep 17 20:49:08.164027 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:08.163980    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:49:08.164457 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:08.163985    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:49:08.164457 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:08.164113    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rb4d7" podUID="c6f7a65c-e785-4e19-aed2-5c2583881c06"
Sep 17 20:49:08.164457 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:08.164223    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhhl5" podUID="b4f3e778-0cec-45fe-be1b-a8789d63c1d7"
Sep 17 20:49:08.164457 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:08.164278    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:49:08.164457 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:08.164363    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r7jbg" podUID="082f604e-de6c-43c3-bd30-02318c0e01a4"
Sep 17 20:49:10.164079 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:10.164049    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:49:10.164504 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:10.164049    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:49:10.164504 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:10.164171    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r7jbg" podUID="082f604e-de6c-43c3-bd30-02318c0e01a4"
Sep 17 20:49:10.164504 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:10.164046    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:49:10.164504 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:10.164250    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rb4d7" podUID="c6f7a65c-e785-4e19-aed2-5c2583881c06"
Sep 17 20:49:10.164504 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:10.164307    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhhl5" podUID="b4f3e778-0cec-45fe-be1b-a8789d63c1d7"
Sep 17 20:49:12.164043 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:12.164007    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:49:12.164441 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:12.164012    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:49:12.164441 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:12.164151    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhhl5" podUID="b4f3e778-0cec-45fe-be1b-a8789d63c1d7"
Sep 17 20:49:12.164441 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:12.164247    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rb4d7" podUID="c6f7a65c-e785-4e19-aed2-5c2583881c06"
Sep 17 20:49:12.164441 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:12.164013    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:49:12.164441 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:12.164352    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r7jbg" podUID="082f604e-de6c-43c3-bd30-02318c0e01a4"
Sep 17 20:49:13.806303 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:13.806264    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-metrics-certs\") pod \"network-metrics-daemon-jhhl5\" (UID: \"b4f3e778-0cec-45fe-be1b-a8789d63c1d7\") " pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:49:13.806303 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:13.806310    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/082f604e-de6c-43c3-bd30-02318c0e01a4-original-pull-secret\") pod \"global-pull-secret-syncer-r7jbg\" (UID: \"082f604e-de6c-43c3-bd30-02318c0e01a4\") " pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:49:13.806962 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:13.806434    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 20:49:13.806962 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:13.806502    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-metrics-certs podName:b4f3e778-0cec-45fe-be1b-a8789d63c1d7 nodeName:}" failed. No retries permitted until 2026-09-17 20:49:29.806483369 +0000 UTC m=+34.152085179 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-metrics-certs") pod "network-metrics-daemon-jhhl5" (UID: "b4f3e778-0cec-45fe-be1b-a8789d63c1d7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 20:49:13.806962 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:13.806435    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 20:49:13.806962 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:13.806592    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/082f604e-de6c-43c3-bd30-02318c0e01a4-original-pull-secret podName:082f604e-de6c-43c3-bd30-02318c0e01a4 nodeName:}" failed. No retries permitted until 2026-09-17 20:49:29.806567235 +0000 UTC m=+34.152169039 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/082f604e-de6c-43c3-bd30-02318c0e01a4-original-pull-secret") pod "global-pull-secret-syncer-r7jbg" (UID: "082f604e-de6c-43c3-bd30-02318c0e01a4") : object "kube-system"/"original-pull-secret" not registered
Sep 17 20:49:13.906978 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:13.906940    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7mbv\" (UniqueName: \"kubernetes.io/projected/c6f7a65c-e785-4e19-aed2-5c2583881c06-kube-api-access-b7mbv\") pod \"network-check-target-rb4d7\" (UID: \"c6f7a65c-e785-4e19-aed2-5c2583881c06\") " pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:49:13.907177 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:13.907116    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 20:49:13.907177 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:13.907132    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 20:49:13.907177 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:13.907142    2567 projected.go:196] Error preparing data for projected volume kube-api-access-b7mbv for pod openshift-network-diagnostics/network-check-target-rb4d7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 20:49:13.907317 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:13.907199    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c6f7a65c-e785-4e19-aed2-5c2583881c06-kube-api-access-b7mbv podName:c6f7a65c-e785-4e19-aed2-5c2583881c06 nodeName:}" failed. No retries permitted until 2026-09-17 20:49:29.907185862 +0000 UTC m=+34.252787668 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-b7mbv" (UniqueName: "kubernetes.io/projected/c6f7a65c-e785-4e19-aed2-5c2583881c06-kube-api-access-b7mbv") pod "network-check-target-rb4d7" (UID: "c6f7a65c-e785-4e19-aed2-5c2583881c06") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 20:49:14.164589 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:14.164503    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:49:14.164771 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:14.164503    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:49:14.164771 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:14.164642    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r7jbg" podUID="082f604e-de6c-43c3-bd30-02318c0e01a4"
Sep 17 20:49:14.164771 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:14.164646    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:49:14.164918 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:14.164768    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhhl5" podUID="b4f3e778-0cec-45fe-be1b-a8789d63c1d7"
Sep 17 20:49:14.164918 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:14.164855    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rb4d7" podUID="c6f7a65c-e785-4e19-aed2-5c2583881c06"
Sep 17 20:49:16.165817 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:16.165786    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:49:16.165817 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:16.165805    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:49:16.166212 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:16.165909    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:49:16.166212 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:16.165916    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhhl5" podUID="b4f3e778-0cec-45fe-be1b-a8789d63c1d7"
Sep 17 20:49:16.166212 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:16.165982    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rb4d7" podUID="c6f7a65c-e785-4e19-aed2-5c2583881c06"
Sep 17 20:49:16.166212 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:16.166048    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r7jbg" podUID="082f604e-de6c-43c3-bd30-02318c0e01a4"
Sep 17 20:49:17.272501 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:17.272265    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-prdg7" event={"ID":"39cff161-a45c-4052-a183-e206fd6360da","Type":"ContainerStarted","Data":"f393614e3961ce3953df28c36ed1f813467da0d7825777d1bc351fed6464b288"}
Sep 17 20:49:17.281639 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:17.281616    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f" event={"ID":"d17553b3-5be3-4200-adec-f293aa7a4f81","Type":"ContainerStarted","Data":"d778e27d97fbd43c17391574f227d2e9efe930c1d5581bf5f8eab543b1945802"}
Sep 17 20:49:17.281743 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:17.281680    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f" event={"ID":"d17553b3-5be3-4200-adec-f293aa7a4f81","Type":"ContainerStarted","Data":"6478419361f87a907a5efd6d3e452464c364c4c9be347835c4be37e15c798de5"}
Sep 17 20:49:17.281743 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:17.281713    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f" event={"ID":"d17553b3-5be3-4200-adec-f293aa7a4f81","Type":"ContainerStarted","Data":"bf85dd3e3a3b406f29625d935bed7086960f6cf364ead3503e6e4523a7d47847"}
Sep 17 20:49:17.281853 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:17.281751    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f" event={"ID":"d17553b3-5be3-4200-adec-f293aa7a4f81","Type":"ContainerStarted","Data":"2aeed68937f609640d98419d8040170eef5bb490f31ae546b9dbdc47b848be94"}
Sep 17 20:49:17.281853 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:17.281801    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f" event={"ID":"d17553b3-5be3-4200-adec-f293aa7a4f81","Type":"ContainerStarted","Data":"cdae5f61fd49f3d4356b0986444b6c2eb74d48a200b8f505274222c3ea9e8249"}
Sep 17 20:49:17.283338 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:17.283313    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4j6w6" event={"ID":"937db66c-5b56-482e-8402-4fd3fe1d388c","Type":"ContainerStarted","Data":"796de603a27ad191e2a9ebe1353d9adfdda19f2485b841b67c31e8e3765c26bc"}
Sep 17 20:49:17.284983 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:17.284961    2567 generic.go:356] "Generic (PLEG): container finished" podID="b1dec440-9d21-409e-a87b-7de07d399656" containerID="faecf44da236c6255719684945fb85c28fbc20ab552b6b0ba7d872f0580749c3" exitCode=0
Sep 17 20:49:17.285072 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:17.285039    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zt8xk" event={"ID":"b1dec440-9d21-409e-a87b-7de07d399656","Type":"ContainerDied","Data":"faecf44da236c6255719684945fb85c28fbc20ab552b6b0ba7d872f0580749c3"}
Sep 17 20:49:17.286606 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:17.286584    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6h956" event={"ID":"6bc4ca39-a164-4ff7-9c1c-fedf495ccb5c","Type":"ContainerStarted","Data":"e91f68e45fabddeeb1e4099f258277117938e0a5eec6eb837f14d3d23a628d38"}
Sep 17 20:49:17.288344 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:17.288326    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4bpjd" event={"ID":"f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b","Type":"ContainerStarted","Data":"a702e81953730ae9bbe77d906a25c5e936715885d568fa432c6724d634ff751c"}
Sep 17 20:49:17.289795 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:17.289775    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-s9k9h" event={"ID":"39f41ed8-5ecd-42b7-8d06-786ad22e316a","Type":"ContainerStarted","Data":"81a3eb3bbd23bcaf46151032148ede2da42dc010ce79a14275949a6588f54484"}
Sep 17 20:49:17.291190 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:17.291169    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5rb9" event={"ID":"a5fe1eef-ed7f-416d-9ff9-c790010c3024","Type":"ContainerStarted","Data":"abbdd1d9a3225b4bf5a53bbb05827c2e35d9e313f0b76b256fe7814f7d02eb02"}
Sep 17 20:49:17.301951 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:17.301913    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-prdg7" podStartSLOduration=4.235998949 podStartE2EDuration="21.30190129s" totalImagesPullingTime="17.065902341s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 20:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:49:17.301707254 +0000 UTC m=+21.647309099" watchObservedRunningTime="2026-09-17 20:49:17.30190129 +0000 UTC m=+21.647503115"
Sep 17 20:49:17.302340 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:17.302313    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-201.ec2.internal" podStartSLOduration=20.30230325 podStartE2EDuration="20.30230325s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 20:48:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:49:01.253925684 +0000 UTC m=+5.599527510" watchObservedRunningTime="2026-09-17 20:49:17.30230325 +0000 UTC m=+21.647905076"
Sep 17 20:49:17.316805 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:17.316742    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6h956" podStartSLOduration=6.921785691 podStartE2EDuration="21.316723576s" totalImagesPullingTime="14.394937885s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 20:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:49:17.316599239 +0000 UTC m=+21.662201089" watchObservedRunningTime="2026-09-17 20:49:17.316723576 +0000 UTC m=+21.662325404"
Sep 17 20:49:17.361904 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:17.361665    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-s9k9h" podStartSLOduration=4.318243867 podStartE2EDuration="21.361648359s" totalImagesPullingTime="17.043404492s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 20:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:49:17.361085759 +0000 UTC m=+21.706687582" watchObservedRunningTime="2026-09-17 20:49:17.361648359 +0000 UTC m=+21.707250213"
Sep 17 20:49:17.376092 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:17.376049    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4j6w6" podStartSLOduration=6.964474318 podStartE2EDuration="21.376030976s" totalImagesPullingTime="14.411556658s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:49:17.375703546 +0000 UTC m=+21.721305371" watchObservedRunningTime="2026-09-17 20:49:17.376030976 +0000 UTC m=+21.721632800"
Sep 17 20:49:17.397416 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:17.397363    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4bpjd" podStartSLOduration=3.283532183 podStartE2EDuration="21.397342828s" totalImagesPullingTime="18.113810645s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 20:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:49:17.39718031 +0000 UTC m=+21.742782155" watchObservedRunningTime="2026-09-17 20:49:17.397342828 +0000 UTC m=+21.742944654"
Sep 17 20:49:18.139580 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:18.139555    2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 17 20:49:18.140472 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:18.140450    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4j6w6"
Sep 17 20:49:18.141176 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:18.141150    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4j6w6"
Sep 17 20:49:18.163952 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:18.163923    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:49:18.164102 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:18.164030    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:49:18.164102 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:18.164045    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:49:18.164102 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:18.164041    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rb4d7" podUID="c6f7a65c-e785-4e19-aed2-5c2583881c06"
Sep 17 20:49:18.164258 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:18.164115    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhhl5" podUID="b4f3e778-0cec-45fe-be1b-a8789d63c1d7"
Sep 17 20:49:18.164319 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:18.164250    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r7jbg" podUID="082f604e-de6c-43c3-bd30-02318c0e01a4"
Sep 17 20:49:18.297224 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:18.297184    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f" event={"ID":"d17553b3-5be3-4200-adec-f293aa7a4f81","Type":"ContainerStarted","Data":"18d21f405494c983c97cffe6030682a936b15574a2362f5574290d1133ecf33b"}
Sep 17 20:49:18.299129 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:18.298852    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-llmv2" event={"ID":"447ec589-f25d-47fb-9713-519a04467695","Type":"ContainerStarted","Data":"e4e42061b1b14fe6e95ea9beb8d0bfadbedd0015a3ea6aa732e6318d7f721aaa"}
Sep 17 20:49:18.301134 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:18.301103    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5rb9" event={"ID":"a5fe1eef-ed7f-416d-9ff9-c790010c3024","Type":"ContainerStarted","Data":"db276148155a47e0b1a0c8b3f9e167bf0fa86f00189dcf85d5471b83fe56fc7a"}
Sep 17 20:49:18.301841 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:18.301808    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4j6w6"
Sep 17 20:49:18.301942 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:18.301923    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4j6w6"
Sep 17 20:49:18.314649 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:18.314608    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-llmv2" podStartSLOduration=5.267005995 podStartE2EDuration="22.314595814s" totalImagesPullingTime="17.047589819s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 20:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:49:18.314394194 +0000 UTC m=+22.659996019" watchObservedRunningTime="2026-09-17 20:49:18.314595814 +0000 UTC m=+22.660197664"
Sep 17 20:49:19.135883 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:19.135688    2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-17T20:49:18.139575127Z","UUID":"a5c7e49b-9253-460c-8b58-bd6f7598df37","Handler":null,"Name":"","Endpoint":""}
Sep 17 20:49:19.137367 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:19.137347    2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 17 20:49:19.137513 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:19.137378    2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 17 20:49:20.164160 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:20.164119    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:49:20.164160 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:20.164138    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:49:20.164668 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:20.164243    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhhl5" podUID="b4f3e778-0cec-45fe-be1b-a8789d63c1d7"
Sep 17 20:49:20.164668 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:20.164285    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r7jbg" podUID="082f604e-de6c-43c3-bd30-02318c0e01a4"
Sep 17 20:49:20.164668 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:20.164335    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:49:20.164668 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:20.164421    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rb4d7" podUID="c6f7a65c-e785-4e19-aed2-5c2583881c06"
Sep 17 20:49:20.307938 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:20.307905    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5rb9" event={"ID":"a5fe1eef-ed7f-416d-9ff9-c790010c3024","Type":"ContainerStarted","Data":"6750ed831932d1077e64cdfa22067234bf2f6126ea8ad0d217882b82061076d9"}
Sep 17 20:49:20.311106 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:20.311081    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f" event={"ID":"d17553b3-5be3-4200-adec-f293aa7a4f81","Type":"ContainerStarted","Data":"64d30484c031b12c76d9411ed0b9a4688e04182f056c3775fefb6e12528c4faa"}
Sep 17 20:49:20.330171 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:20.330136    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5rb9" podStartSLOduration=4.902827481 podStartE2EDuration="24.330122424s" totalImagesPullingTime="19.427294943s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 20:48:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:49:20.330060925 +0000 UTC m=+24.675662762" watchObservedRunningTime="2026-09-17 20:49:20.330122424 +0000 UTC m=+24.675724248"
Sep 17 20:49:22.164447 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:22.164231    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:49:22.164937 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:22.164231    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:49:22.164937 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:22.164554    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rb4d7" podUID="c6f7a65c-e785-4e19-aed2-5c2583881c06"
Sep 17 20:49:22.164937 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:22.164288    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:49:22.164937 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:22.164615    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r7jbg" podUID="082f604e-de6c-43c3-bd30-02318c0e01a4"
Sep 17 20:49:22.164937 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:22.164727    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhhl5" podUID="b4f3e778-0cec-45fe-be1b-a8789d63c1d7"
Sep 17 20:49:23.321648 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:23.321610    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f" event={"ID":"d17553b3-5be3-4200-adec-f293aa7a4f81","Type":"ContainerStarted","Data":"f0680f974800286d1d16418a53ee5a2437741f2ce636f8822d777dee18099bc9"}
Sep 17 20:49:23.322537 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:23.321881    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:49:23.322537 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:23.321951    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:49:23.322537 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:23.321994    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:49:23.323748 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:23.323721    2567 generic.go:356] "Generic (PLEG): container finished" podID="b1dec440-9d21-409e-a87b-7de07d399656" containerID="5d0ac76b75b766b0aa694e0001b1d560b3adc4b85f89e0c5fb0b4c9ae401dc2a" exitCode=0
Sep 17 20:49:23.323874 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:23.323792    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zt8xk" event={"ID":"b1dec440-9d21-409e-a87b-7de07d399656","Type":"ContainerDied","Data":"5d0ac76b75b766b0aa694e0001b1d560b3adc4b85f89e0c5fb0b4c9ae401dc2a"}
Sep 17 20:49:23.337309 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:23.337290    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:49:23.337399 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:23.337377    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:49:23.352822 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:23.352782    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f" podStartSLOduration=9.027028036 podStartE2EDuration="27.352746625s" totalImagesPullingTime="18.325718589s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 20:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:49:23.352218786 +0000 UTC m=+27.697820611" watchObservedRunningTime="2026-09-17 20:49:23.352746625 +0000 UTC m=+27.698348450"
Sep 17 20:49:24.164502 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:24.164323    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:49:24.164670 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:24.164323    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:49:24.164670 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:24.164588    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhhl5" podUID="b4f3e778-0cec-45fe-be1b-a8789d63c1d7"
Sep 17 20:49:24.164670 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:24.164389    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:49:24.164670 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:24.164647    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r7jbg" podUID="082f604e-de6c-43c3-bd30-02318c0e01a4"
Sep 17 20:49:24.164935 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:24.164721    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rb4d7" podUID="c6f7a65c-e785-4e19-aed2-5c2583881c06"
Sep 17 20:49:24.274697 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:24.274665    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-r7jbg"]
Sep 17 20:49:24.278317 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:24.278290    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rb4d7"]
Sep 17 20:49:24.290708 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:24.290682    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jhhl5"]
Sep 17 20:49:24.325776 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:24.325739    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:49:24.326100 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:24.325737    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:49:24.326100 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:24.325861    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhhl5" podUID="b4f3e778-0cec-45fe-be1b-a8789d63c1d7"
Sep 17 20:49:24.326100 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:24.325915    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r7jbg" podUID="082f604e-de6c-43c3-bd30-02318c0e01a4"
Sep 17 20:49:24.326100 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:24.325739    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:49:24.326294 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:24.326275    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rb4d7" podUID="c6f7a65c-e785-4e19-aed2-5c2583881c06"
Sep 17 20:49:25.329935 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:25.329903    2567 generic.go:356] "Generic (PLEG): container finished" podID="b1dec440-9d21-409e-a87b-7de07d399656" containerID="5c4d66096864b3bd2e316bdeb30b9161325036e9492909cdf098535c948078bf" exitCode=0
Sep 17 20:49:25.330278 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:25.329990    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zt8xk" event={"ID":"b1dec440-9d21-409e-a87b-7de07d399656","Type":"ContainerDied","Data":"5c4d66096864b3bd2e316bdeb30b9161325036e9492909cdf098535c948078bf"}
Sep 17 20:49:26.165578 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:26.165547    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:49:26.165724 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:26.165650    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:49:26.165724 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:26.165689    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:49:26.165724 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:26.165688    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhhl5" podUID="b4f3e778-0cec-45fe-be1b-a8789d63c1d7"
Sep 17 20:49:26.165861 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:26.165789    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rb4d7" podUID="c6f7a65c-e785-4e19-aed2-5c2583881c06"
Sep 17 20:49:26.165861 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:26.165840    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r7jbg" podUID="082f604e-de6c-43c3-bd30-02318c0e01a4"
Sep 17 20:49:27.336174 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:27.336140    2567 generic.go:356] "Generic (PLEG): container finished" podID="b1dec440-9d21-409e-a87b-7de07d399656" containerID="774749c9a05ca9de1b7e166d048c1cb3297575ceac7b11fcfa589a8c7367e585" exitCode=0
Sep 17 20:49:27.336174 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:27.336180    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zt8xk" event={"ID":"b1dec440-9d21-409e-a87b-7de07d399656","Type":"ContainerDied","Data":"774749c9a05ca9de1b7e166d048c1cb3297575ceac7b11fcfa589a8c7367e585"}
Sep 17 20:49:28.164845 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:28.164649    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:49:28.165003 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:28.164650    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:49:28.165003 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:28.164915    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rb4d7" podUID="c6f7a65c-e785-4e19-aed2-5c2583881c06"
Sep 17 20:49:28.165003 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:28.164661    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:49:28.165172 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:28.165140    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhhl5" podUID="b4f3e778-0cec-45fe-be1b-a8789d63c1d7"
Sep 17 20:49:28.165172 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:28.164995    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r7jbg" podUID="082f604e-de6c-43c3-bd30-02318c0e01a4"
Sep 17 20:49:29.821163 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:29.821124    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-metrics-certs\") pod \"network-metrics-daemon-jhhl5\" (UID: \"b4f3e778-0cec-45fe-be1b-a8789d63c1d7\") " pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:49:29.821657 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:29.821172    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/082f604e-de6c-43c3-bd30-02318c0e01a4-original-pull-secret\") pod \"global-pull-secret-syncer-r7jbg\" (UID: \"082f604e-de6c-43c3-bd30-02318c0e01a4\") " pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:49:29.821657 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:29.821258    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 20:49:29.821657 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:29.821260    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 20:49:29.821657 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:29.821311    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/082f604e-de6c-43c3-bd30-02318c0e01a4-original-pull-secret podName:082f604e-de6c-43c3-bd30-02318c0e01a4 nodeName:}" failed. No retries permitted until 2026-09-17 20:50:01.821297219 +0000 UTC m=+66.166899026 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/082f604e-de6c-43c3-bd30-02318c0e01a4-original-pull-secret") pod "global-pull-secret-syncer-r7jbg" (UID: "082f604e-de6c-43c3-bd30-02318c0e01a4") : object "kube-system"/"original-pull-secret" not registered
Sep 17 20:49:29.821657 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:29.821324    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-metrics-certs podName:b4f3e778-0cec-45fe-be1b-a8789d63c1d7 nodeName:}" failed. No retries permitted until 2026-09-17 20:50:01.821317776 +0000 UTC m=+66.166919579 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-metrics-certs") pod "network-metrics-daemon-jhhl5" (UID: "b4f3e778-0cec-45fe-be1b-a8789d63c1d7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 20:49:29.921779 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:29.921687    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7mbv\" (UniqueName: \"kubernetes.io/projected/c6f7a65c-e785-4e19-aed2-5c2583881c06-kube-api-access-b7mbv\") pod \"network-check-target-rb4d7\" (UID: \"c6f7a65c-e785-4e19-aed2-5c2583881c06\") " pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:49:29.921921 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:29.921869    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 20:49:29.921921 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:29.921890    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 20:49:29.921921 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:29.921899    2567 projected.go:196] Error preparing data for projected volume kube-api-access-b7mbv for pod openshift-network-diagnostics/network-check-target-rb4d7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 20:49:29.922036 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:29.921954    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c6f7a65c-e785-4e19-aed2-5c2583881c06-kube-api-access-b7mbv podName:c6f7a65c-e785-4e19-aed2-5c2583881c06 nodeName:}" failed. No retries permitted until 2026-09-17 20:50:01.92193816 +0000 UTC m=+66.267539968 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-b7mbv" (UniqueName: "kubernetes.io/projected/c6f7a65c-e785-4e19-aed2-5c2583881c06-kube-api-access-b7mbv") pod "network-check-target-rb4d7" (UID: "c6f7a65c-e785-4e19-aed2-5c2583881c06") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 20:49:30.164142 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.164104    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:49:30.164328 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.164111    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:49:30.164328 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.164238    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:49:30.164328 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:30.164241    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rb4d7" podUID="c6f7a65c-e785-4e19-aed2-5c2583881c06"
Sep 17 20:49:30.164490 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:30.164323    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r7jbg" podUID="082f604e-de6c-43c3-bd30-02318c0e01a4"
Sep 17 20:49:30.164544 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:30.164482    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhhl5" podUID="b4f3e778-0cec-45fe-be1b-a8789d63c1d7"
Sep 17 20:49:30.490041 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.490010    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-201.ec2.internal" event="NodeReady"
Sep 17 20:49:30.490215 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.490158    2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 17 20:49:30.528957 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.528916    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-854db97696-6sllh"]
Sep 17 20:49:30.556284 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.556254    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-854db97696-6sllh"]
Sep 17 20:49:30.556284 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.556293    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-b8zb8"]
Sep 17 20:49:30.556520 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.556411    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:49:30.559020 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.558997    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 17 20:49:30.559164 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.559141    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 17 20:49:30.559236 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.559168    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 17 20:49:30.559236 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.559175    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9cn4m\""
Sep 17 20:49:30.571790 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.571748    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 17 20:49:30.572243 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.572223    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9v7ql"]
Sep 17 20:49:30.572421 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.572386    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b8zb8"
Sep 17 20:49:30.575035 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.575004    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 17 20:49:30.575035 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.575024    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 17 20:49:30.575186 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.575030    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 17 20:49:30.575186 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.575119    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7zl29\""
Sep 17 20:49:30.588691 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.588666    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b8zb8"]
Sep 17 20:49:30.588834 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.588709    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9v7ql"
Sep 17 20:49:30.588834 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.588768    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9v7ql"]
Sep 17 20:49:30.591416 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.591392    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 17 20:49:30.591515 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.591451    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 17 20:49:30.591515 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.591474    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lzdg7\""
Sep 17 20:49:30.628739 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.628708    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bg2bm\" (UniqueName: \"kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-kube-api-access-bg2bm\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:49:30.628911 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.628747    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/662880e7-4dd4-4556-a2db-ba6b415784aa-ca-trust-extracted\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:49:30.628911 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.628814    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-tls\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:49:30.628911 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.628837    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-certificates\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:49:30.628911 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.628860    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-bound-sa-token\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:49:30.629068 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.628938    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/662880e7-4dd4-4556-a2db-ba6b415784aa-trusted-ca\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:49:30.629068 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.628992    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/662880e7-4dd4-4556-a2db-ba6b415784aa-image-registry-private-configuration\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:49:30.629068 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.629023    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/662880e7-4dd4-4556-a2db-ba6b415784aa-installation-pull-secrets\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:49:30.719767 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.719705    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:30.729489 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.729460    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/662880e7-4dd4-4556-a2db-ba6b415784aa-trusted-ca\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:49:30.729657 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.729514    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60ba7521-2f26-41e6-a17c-98e48bf83c94-tmp-dir\") pod \"dns-default-9v7ql\" (UID: \"60ba7521-2f26-41e6-a17c-98e48bf83c94\") " pod="openshift-dns/dns-default-9v7ql"
Sep 17 20:49:30.729657 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.729543    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kndb9\" (UniqueName: \"kubernetes.io/projected/60ba7521-2f26-41e6-a17c-98e48bf83c94-kube-api-access-kndb9\") pod \"dns-default-9v7ql\" (UID: \"60ba7521-2f26-41e6-a17c-98e48bf83c94\") " pod="openshift-dns/dns-default-9v7ql"
Sep 17 20:49:30.729657 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.729626    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/662880e7-4dd4-4556-a2db-ba6b415784aa-image-registry-private-configuration\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:49:30.729657 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.729653    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/662880e7-4dd4-4556-a2db-ba6b415784aa-installation-pull-secrets\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:49:30.729918 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.729676    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bg2bm\" (UniqueName: \"kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-kube-api-access-bg2bm\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:49:30.729918 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.729704    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdk6j\" (UniqueName: \"kubernetes.io/projected/aa9e55ff-3a8d-477b-8165-ce0c5183b15f-kube-api-access-vdk6j\") pod \"ingress-canary-b8zb8\" (UID: \"aa9e55ff-3a8d-477b-8165-ce0c5183b15f\") " pod="openshift-ingress-canary/ingress-canary-b8zb8"
Sep 17 20:49:30.729918 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.729735    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/662880e7-4dd4-4556-a2db-ba6b415784aa-ca-trust-extracted\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:49:30.729918 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.729811    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-tls\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:49:30.729918 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.729831    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/60ba7521-2f26-41e6-a17c-98e48bf83c94-config-volume\") pod \"dns-default-9v7ql\" (UID: \"60ba7521-2f26-41e6-a17c-98e48bf83c94\") " pod="openshift-dns/dns-default-9v7ql"
Sep 17 20:49:30.729918 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.729857    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-certificates\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:49:30.729918 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.729881    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-bound-sa-token\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:49:30.730249 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.729938    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60ba7521-2f26-41e6-a17c-98e48bf83c94-metrics-tls\") pod \"dns-default-9v7ql\" (UID: \"60ba7521-2f26-41e6-a17c-98e48bf83c94\") " pod="openshift-dns/dns-default-9v7ql"
Sep 17 20:49:30.730249 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:30.730033    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 20:49:30.730249 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:30.730046    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-854db97696-6sllh: secret "image-registry-tls" not found
Sep 17 20:49:30.730249 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:30.730100    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-tls podName:662880e7-4dd4-4556-a2db-ba6b415784aa nodeName:}" failed. No retries permitted until 2026-09-17 20:49:31.230080197 +0000 UTC m=+35.575682003 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-tls") pod "image-registry-854db97696-6sllh" (UID: "662880e7-4dd4-4556-a2db-ba6b415784aa") : secret "image-registry-tls" not found
Sep 17 20:49:30.730249 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.730151    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/662880e7-4dd4-4556-a2db-ba6b415784aa-ca-trust-extracted\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:49:30.730249 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.730195    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa9e55ff-3a8d-477b-8165-ce0c5183b15f-cert\") pod \"ingress-canary-b8zb8\" (UID: \"aa9e55ff-3a8d-477b-8165-ce0c5183b15f\") " pod="openshift-ingress-canary/ingress-canary-b8zb8"
Sep 17 20:49:30.730556 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.730444    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-certificates\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:49:30.730611 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.730546    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/662880e7-4dd4-4556-a2db-ba6b415784aa-trusted-ca\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:49:30.734802 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.734782    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/662880e7-4dd4-4556-a2db-ba6b415784aa-image-registry-private-configuration\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:49:30.734909 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.734798    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/662880e7-4dd4-4556-a2db-ba6b415784aa-installation-pull-secrets\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:49:30.741557 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.741496    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-bound-sa-token\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:49:30.741774 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.741737    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bg2bm\" (UniqueName: \"kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-kube-api-access-bg2bm\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:49:30.811066 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.811031    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:30.830844 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.830812    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/60ba7521-2f26-41e6-a17c-98e48bf83c94-config-volume\") pod \"dns-default-9v7ql\" (UID: \"60ba7521-2f26-41e6-a17c-98e48bf83c94\") " pod="openshift-dns/dns-default-9v7ql"
Sep 17 20:49:30.831522 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.830861    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60ba7521-2f26-41e6-a17c-98e48bf83c94-metrics-tls\") pod \"dns-default-9v7ql\" (UID: \"60ba7521-2f26-41e6-a17c-98e48bf83c94\") " pod="openshift-dns/dns-default-9v7ql"
Sep 17 20:49:30.831522 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:30.830978    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 20:49:30.831522 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.830998    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa9e55ff-3a8d-477b-8165-ce0c5183b15f-cert\") pod \"ingress-canary-b8zb8\" (UID: \"aa9e55ff-3a8d-477b-8165-ce0c5183b15f\") " pod="openshift-ingress-canary/ingress-canary-b8zb8"
Sep 17 20:49:30.831522 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:30.831041    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60ba7521-2f26-41e6-a17c-98e48bf83c94-metrics-tls podName:60ba7521-2f26-41e6-a17c-98e48bf83c94 nodeName:}" failed. No retries permitted until 2026-09-17 20:49:31.331020724 +0000 UTC m=+35.676622546 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60ba7521-2f26-41e6-a17c-98e48bf83c94-metrics-tls") pod "dns-default-9v7ql" (UID: "60ba7521-2f26-41e6-a17c-98e48bf83c94") : secret "dns-default-metrics-tls" not found
Sep 17 20:49:30.831522 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.831084    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60ba7521-2f26-41e6-a17c-98e48bf83c94-tmp-dir\") pod \"dns-default-9v7ql\" (UID: \"60ba7521-2f26-41e6-a17c-98e48bf83c94\") " pod="openshift-dns/dns-default-9v7ql"
Sep 17 20:49:30.831522 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.831109    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kndb9\" (UniqueName: \"kubernetes.io/projected/60ba7521-2f26-41e6-a17c-98e48bf83c94-kube-api-access-kndb9\") pod \"dns-default-9v7ql\" (UID: \"60ba7521-2f26-41e6-a17c-98e48bf83c94\") " pod="openshift-dns/dns-default-9v7ql"
Sep 17 20:49:30.831522 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.831166    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdk6j\" (UniqueName: \"kubernetes.io/projected/aa9e55ff-3a8d-477b-8165-ce0c5183b15f-kube-api-access-vdk6j\") pod \"ingress-canary-b8zb8\" (UID: \"aa9e55ff-3a8d-477b-8165-ce0c5183b15f\") " pod="openshift-ingress-canary/ingress-canary-b8zb8"
Sep 17 20:49:30.831522 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.831404    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60ba7521-2f26-41e6-a17c-98e48bf83c94-tmp-dir\") pod \"dns-default-9v7ql\" (UID: \"60ba7521-2f26-41e6-a17c-98e48bf83c94\") " pod="openshift-dns/dns-default-9v7ql"
Sep 17 20:49:30.831522 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.831465    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/60ba7521-2f26-41e6-a17c-98e48bf83c94-config-volume\") pod \"dns-default-9v7ql\" (UID: \"60ba7521-2f26-41e6-a17c-98e48bf83c94\") " pod="openshift-dns/dns-default-9v7ql"
Sep 17 20:49:30.833655 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.833631    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa9e55ff-3a8d-477b-8165-ce0c5183b15f-cert\") pod \"ingress-canary-b8zb8\" (UID: \"aa9e55ff-3a8d-477b-8165-ce0c5183b15f\") " pod="openshift-ingress-canary/ingress-canary-b8zb8"
Sep 17 20:49:30.840735 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.840714    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kndb9\" (UniqueName: \"kubernetes.io/projected/60ba7521-2f26-41e6-a17c-98e48bf83c94-kube-api-access-kndb9\") pod \"dns-default-9v7ql\" (UID: \"60ba7521-2f26-41e6-a17c-98e48bf83c94\") " pod="openshift-dns/dns-default-9v7ql"
Sep 17 20:49:30.841455 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.841438    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdk6j\" (UniqueName: \"kubernetes.io/projected/aa9e55ff-3a8d-477b-8165-ce0c5183b15f-kube-api-access-vdk6j\") pod \"ingress-canary-b8zb8\" (UID: \"aa9e55ff-3a8d-477b-8165-ce0c5183b15f\") " pod="openshift-ingress-canary/ingress-canary-b8zb8"
Sep 17 20:49:30.882115 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:30.882083    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b8zb8"
Sep 17 20:49:31.234365 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:31.234333    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-tls\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:49:31.234537 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:31.234470    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 20:49:31.234537 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:31.234485    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-854db97696-6sllh: secret "image-registry-tls" not found
Sep 17 20:49:31.234537 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:31.234536    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-tls podName:662880e7-4dd4-4556-a2db-ba6b415784aa nodeName:}" failed. No retries permitted until 2026-09-17 20:49:32.23452313 +0000 UTC m=+36.580124933 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-tls") pod "image-registry-854db97696-6sllh" (UID: "662880e7-4dd4-4556-a2db-ba6b415784aa") : secret "image-registry-tls" not found
Sep 17 20:49:31.335200 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:31.335158    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60ba7521-2f26-41e6-a17c-98e48bf83c94-metrics-tls\") pod \"dns-default-9v7ql\" (UID: \"60ba7521-2f26-41e6-a17c-98e48bf83c94\") " pod="openshift-dns/dns-default-9v7ql"
Sep 17 20:49:31.335375 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:31.335314    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 20:49:31.335435 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:31.335379    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60ba7521-2f26-41e6-a17c-98e48bf83c94-metrics-tls podName:60ba7521-2f26-41e6-a17c-98e48bf83c94 nodeName:}" failed. No retries permitted until 2026-09-17 20:49:32.335359528 +0000 UTC m=+36.680961337 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60ba7521-2f26-41e6-a17c-98e48bf83c94-metrics-tls") pod "dns-default-9v7ql" (UID: "60ba7521-2f26-41e6-a17c-98e48bf83c94") : secret "dns-default-metrics-tls" not found
Sep 17 20:49:31.734841 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:31.734810    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b8zb8"]
Sep 17 20:49:31.738272 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:49:31.738248    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa9e55ff_3a8d_477b_8165_ce0c5183b15f.slice/crio-38deb9c5ad6edd0bd675173c1b435ce5c5211945cf59d7b9954bcae823848a4f WatchSource:0}: Error finding container 38deb9c5ad6edd0bd675173c1b435ce5c5211945cf59d7b9954bcae823848a4f: Status 404 returned error can't find the container with id 38deb9c5ad6edd0bd675173c1b435ce5c5211945cf59d7b9954bcae823848a4f
Sep 17 20:49:32.164336 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:32.164092    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:49:32.164997 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:32.164097    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:49:32.164997 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:32.164109    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:49:32.167269 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:32.167249    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-w6qzl\""
Sep 17 20:49:32.168377 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:32.168352    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 20:49:32.168452 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:32.168395    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 20:49:32.168504 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:32.168447    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 20:49:32.168504 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:32.168398    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 20:49:32.168579 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:32.168533    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6gfd5\""
Sep 17 20:49:32.241118 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:32.241084    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-tls\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:49:32.241286 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:32.241255    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 20:49:32.241286 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:32.241274    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-854db97696-6sllh: secret "image-registry-tls" not found
Sep 17 20:49:32.241394 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:32.241346    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-tls podName:662880e7-4dd4-4556-a2db-ba6b415784aa nodeName:}" failed. No retries permitted until 2026-09-17 20:49:34.241325864 +0000 UTC m=+38.586927690 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-tls") pod "image-registry-854db97696-6sllh" (UID: "662880e7-4dd4-4556-a2db-ba6b415784aa") : secret "image-registry-tls" not found
Sep 17 20:49:32.342479 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:32.342442    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60ba7521-2f26-41e6-a17c-98e48bf83c94-metrics-tls\") pod \"dns-default-9v7ql\" (UID: \"60ba7521-2f26-41e6-a17c-98e48bf83c94\") " pod="openshift-dns/dns-default-9v7ql"
Sep 17 20:49:32.342658 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:32.342624    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 20:49:32.342713 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:32.342704    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60ba7521-2f26-41e6-a17c-98e48bf83c94-metrics-tls podName:60ba7521-2f26-41e6-a17c-98e48bf83c94 nodeName:}" failed. No retries permitted until 2026-09-17 20:49:34.342683962 +0000 UTC m=+38.688285766 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60ba7521-2f26-41e6-a17c-98e48bf83c94-metrics-tls") pod "dns-default-9v7ql" (UID: "60ba7521-2f26-41e6-a17c-98e48bf83c94") : secret "dns-default-metrics-tls" not found
Sep 17 20:49:32.352933 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:32.352902    2567 generic.go:356] "Generic (PLEG): container finished" podID="b1dec440-9d21-409e-a87b-7de07d399656" containerID="5507085dabc569fbb1f2639f74e06001b13b6605b430cb983caf77dfcf009ce5" exitCode=0
Sep 17 20:49:32.353069 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:32.352999    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zt8xk" event={"ID":"b1dec440-9d21-409e-a87b-7de07d399656","Type":"ContainerDied","Data":"5507085dabc569fbb1f2639f74e06001b13b6605b430cb983caf77dfcf009ce5"}
Sep 17 20:49:32.354401 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:32.354375    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b8zb8" event={"ID":"aa9e55ff-3a8d-477b-8165-ce0c5183b15f","Type":"ContainerStarted","Data":"38deb9c5ad6edd0bd675173c1b435ce5c5211945cf59d7b9954bcae823848a4f"}
Sep 17 20:49:33.361296 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:33.361264    2567 generic.go:356] "Generic (PLEG): container finished" podID="b1dec440-9d21-409e-a87b-7de07d399656" containerID="4b25f5f7e6bd41fe4761b1d0fd358fabd5d3bb6c74c922f708c6c8b743483e84" exitCode=0
Sep 17 20:49:33.361670 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:33.361312    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zt8xk" event={"ID":"b1dec440-9d21-409e-a87b-7de07d399656","Type":"ContainerDied","Data":"4b25f5f7e6bd41fe4761b1d0fd358fabd5d3bb6c74c922f708c6c8b743483e84"}
Sep 17 20:49:34.259679 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:34.259642    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-tls\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:49:34.259840 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:34.259817    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 20:49:34.259840 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:34.259839    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-854db97696-6sllh: secret "image-registry-tls" not found
Sep 17 20:49:34.259934 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:34.259899    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-tls podName:662880e7-4dd4-4556-a2db-ba6b415784aa nodeName:}" failed. No retries permitted until 2026-09-17 20:49:38.259880323 +0000 UTC m=+42.605482126 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-tls") pod "image-registry-854db97696-6sllh" (UID: "662880e7-4dd4-4556-a2db-ba6b415784aa") : secret "image-registry-tls" not found
Sep 17 20:49:34.360908 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:34.360885    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60ba7521-2f26-41e6-a17c-98e48bf83c94-metrics-tls\") pod \"dns-default-9v7ql\" (UID: \"60ba7521-2f26-41e6-a17c-98e48bf83c94\") " pod="openshift-dns/dns-default-9v7ql"
Sep 17 20:49:34.361043 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:34.361027    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 20:49:34.361098 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:34.361087    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60ba7521-2f26-41e6-a17c-98e48bf83c94-metrics-tls podName:60ba7521-2f26-41e6-a17c-98e48bf83c94 nodeName:}" failed. No retries permitted until 2026-09-17 20:49:38.361073222 +0000 UTC m=+42.706675025 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60ba7521-2f26-41e6-a17c-98e48bf83c94-metrics-tls") pod "dns-default-9v7ql" (UID: "60ba7521-2f26-41e6-a17c-98e48bf83c94") : secret "dns-default-metrics-tls" not found
Sep 17 20:49:34.366360 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:34.366339    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zt8xk" event={"ID":"b1dec440-9d21-409e-a87b-7de07d399656","Type":"ContainerStarted","Data":"0cea45791149966894e2f08f87501408ebba72f432999a1f63d47da3441231f0"}
Sep 17 20:49:34.408643 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:34.408599    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zt8xk" podStartSLOduration=9.482122301 podStartE2EDuration="38.408584491s" totalImagesPullingTime="28.92646219s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 20:48:56 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:49:34.407344035 +0000 UTC m=+38.752945895" watchObservedRunningTime="2026-09-17 20:49:34.408584491 +0000 UTC m=+38.754186316"
Sep 17 20:49:34.533211 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:34.533172    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:34.630411 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:34.630320    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:34.947974 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:34.947734    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:35.009720 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:35.009683    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:35.369956 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:35.369917    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b8zb8" event={"ID":"aa9e55ff-3a8d-477b-8165-ce0c5183b15f","Type":"ContainerStarted","Data":"a32549c3b10946380fd5c4f741d51f9344f22f3502def27e11c8c9dd94d415ea"}
Sep 17 20:49:35.387547 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:35.387500    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-b8zb8" podStartSLOduration=2.766983377 podStartE2EDuration="5.387486868s" totalImagesPullingTime="2.620503491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:49:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:49:35.386810702 +0000 UTC m=+39.732412552" watchObservedRunningTime="2026-09-17 20:49:35.387486868 +0000 UTC m=+39.733088692"
Sep 17 20:49:38.289108 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:38.289067    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-tls\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:49:38.289504 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:38.289213    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 20:49:38.289504 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:38.289233    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-854db97696-6sllh: secret "image-registry-tls" not found
Sep 17 20:49:38.289504 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:38.289283    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-tls podName:662880e7-4dd4-4556-a2db-ba6b415784aa nodeName:}" failed. No retries permitted until 2026-09-17 20:49:46.289270056 +0000 UTC m=+50.634871858 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-tls") pod "image-registry-854db97696-6sllh" (UID: "662880e7-4dd4-4556-a2db-ba6b415784aa") : secret "image-registry-tls" not found
Sep 17 20:49:38.390354 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:38.390318    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60ba7521-2f26-41e6-a17c-98e48bf83c94-metrics-tls\") pod \"dns-default-9v7ql\" (UID: \"60ba7521-2f26-41e6-a17c-98e48bf83c94\") " pod="openshift-dns/dns-default-9v7ql"
Sep 17 20:49:38.390513 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:38.390473    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 20:49:38.390560 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:38.390531    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60ba7521-2f26-41e6-a17c-98e48bf83c94-metrics-tls podName:60ba7521-2f26-41e6-a17c-98e48bf83c94 nodeName:}" failed. No retries permitted until 2026-09-17 20:49:46.39051654 +0000 UTC m=+50.736118343 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60ba7521-2f26-41e6-a17c-98e48bf83c94-metrics-tls") pod "dns-default-9v7ql" (UID: "60ba7521-2f26-41e6-a17c-98e48bf83c94") : secret "dns-default-metrics-tls" not found
Sep 17 20:49:41.685168 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:41.685131    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:41.756799 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:41.756746    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:42.418657 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:42.418624    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:42.492929 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:42.492896    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:42.612646 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:42.612613    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:42.827518 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:42.827487    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:43.036534 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:43.036501    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:43.115073 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:43.114990    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:43.770618 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:43.770573    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:43.842549 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:43.842523    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:44.044857 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:44.044753    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:44.429732 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:44.429640    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:44.534345 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:44.534315    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:46.346952 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:46.346912    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-tls\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:49:46.347441 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:46.347068    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 20:49:46.347441 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:46.347086    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-854db97696-6sllh: secret "image-registry-tls" not found
Sep 17 20:49:46.347441 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:46.347142    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-tls podName:662880e7-4dd4-4556-a2db-ba6b415784aa nodeName:}" failed. No retries permitted until 2026-09-17 20:50:02.347127202 +0000 UTC m=+66.692729011 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-tls") pod "image-registry-854db97696-6sllh" (UID: "662880e7-4dd4-4556-a2db-ba6b415784aa") : secret "image-registry-tls" not found
Sep 17 20:49:46.447743 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:46.447697    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60ba7521-2f26-41e6-a17c-98e48bf83c94-metrics-tls\") pod \"dns-default-9v7ql\" (UID: \"60ba7521-2f26-41e6-a17c-98e48bf83c94\") " pod="openshift-dns/dns-default-9v7ql"
Sep 17 20:49:46.447922 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:46.447862    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 20:49:46.447962 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:49:46.447932    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60ba7521-2f26-41e6-a17c-98e48bf83c94-metrics-tls podName:60ba7521-2f26-41e6-a17c-98e48bf83c94 nodeName:}" failed. No retries permitted until 2026-09-17 20:50:02.447917163 +0000 UTC m=+66.793518966 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60ba7521-2f26-41e6-a17c-98e48bf83c94-metrics-tls") pod "dns-default-9v7ql" (UID: "60ba7521-2f26-41e6-a17c-98e48bf83c94") : secret "dns-default-metrics-tls" not found
Sep 17 20:49:48.704906 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:48.704874    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:53.618217 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:53.618182    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:53.736104 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:53.736075    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:54.757111 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:54.757080    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:54.864437 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:54.864404    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:55.340689 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:55.340663    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gcr8f"
Sep 17 20:49:55.498790 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:55.498740    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:55.570226 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:55.570192    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:55.645109 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:55.645040    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:57.814617 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:57.814578    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:57.905461 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:57.905430    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:58.902593 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:58.902558    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:59.034593 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:59.034560    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:59.607344 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:59.607303    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:59.752568 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:49:59.752532    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:00.080942 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:00.080906    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:00.163374 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:00.163330    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:01.866682 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:01.866637    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-metrics-certs\") pod \"network-metrics-daemon-jhhl5\" (UID: \"b4f3e778-0cec-45fe-be1b-a8789d63c1d7\") " pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:50:01.866682 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:01.866691    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/082f604e-de6c-43c3-bd30-02318c0e01a4-original-pull-secret\") pod \"global-pull-secret-syncer-r7jbg\" (UID: \"082f604e-de6c-43c3-bd30-02318c0e01a4\") " pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:50:01.869538 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:01.869511    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 20:50:01.869654 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:01.869542    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 20:50:01.877009 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:50:01.876988    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 17 20:50:01.877081 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:50:01.877069    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-metrics-certs podName:b4f3e778-0cec-45fe-be1b-a8789d63c1d7 nodeName:}" failed. No retries permitted until 2026-09-17 20:51:05.877049976 +0000 UTC m=+130.222651793 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-metrics-certs") pod "network-metrics-daemon-jhhl5" (UID: "b4f3e778-0cec-45fe-be1b-a8789d63c1d7") : secret "metrics-daemon-secret" not found
Sep 17 20:50:01.880509 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:01.880483    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/082f604e-de6c-43c3-bd30-02318c0e01a4-original-pull-secret\") pod \"global-pull-secret-syncer-r7jbg\" (UID: \"082f604e-de6c-43c3-bd30-02318c0e01a4\") " pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:50:01.884444 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:01.884427    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r7jbg"
Sep 17 20:50:01.967296 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:01.967261    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7mbv\" (UniqueName: \"kubernetes.io/projected/c6f7a65c-e785-4e19-aed2-5c2583881c06-kube-api-access-b7mbv\") pod \"network-check-target-rb4d7\" (UID: \"c6f7a65c-e785-4e19-aed2-5c2583881c06\") " pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:50:01.970409 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:01.970389    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 20:50:01.980949 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:01.980929    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 20:50:01.990885 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:01.990863    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7mbv\" (UniqueName: \"kubernetes.io/projected/c6f7a65c-e785-4e19-aed2-5c2583881c06-kube-api-access-b7mbv\") pod \"network-check-target-rb4d7\" (UID: \"c6f7a65c-e785-4e19-aed2-5c2583881c06\") " pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:50:02.167141 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:02.167075    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-r7jbg"]
Sep 17 20:50:02.170126 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:50:02.170101    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod082f604e_de6c_43c3_bd30_02318c0e01a4.slice/crio-21a857e18ebf27d073336b0e8f72545a9ab974eb47210588d98ce6a250deaff1 WatchSource:0}: Error finding container 21a857e18ebf27d073336b0e8f72545a9ab974eb47210588d98ce6a250deaff1: Status 404 returned error can't find the container with id 21a857e18ebf27d073336b0e8f72545a9ab974eb47210588d98ce6a250deaff1
Sep 17 20:50:02.180277 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:02.180255    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-w6qzl\""
Sep 17 20:50:02.185333 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:02.185309    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:50:02.368559 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:02.368520    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-tls\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:50:02.368742 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:50:02.368643    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 20:50:02.368742 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:50:02.368657    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-854db97696-6sllh: secret "image-registry-tls" not found
Sep 17 20:50:02.368742 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:50:02.368706    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-tls podName:662880e7-4dd4-4556-a2db-ba6b415784aa nodeName:}" failed. No retries permitted until 2026-09-17 20:50:34.368693167 +0000 UTC m=+98.714294970 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-tls") pod "image-registry-854db97696-6sllh" (UID: "662880e7-4dd4-4556-a2db-ba6b415784aa") : secret "image-registry-tls" not found
Sep 17 20:50:02.438494 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:02.438413    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-r7jbg" event={"ID":"082f604e-de6c-43c3-bd30-02318c0e01a4","Type":"ContainerStarted","Data":"21a857e18ebf27d073336b0e8f72545a9ab974eb47210588d98ce6a250deaff1"}
Sep 17 20:50:02.467260 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:02.467232    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rb4d7"]
Sep 17 20:50:02.469630 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:02.469608    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60ba7521-2f26-41e6-a17c-98e48bf83c94-metrics-tls\") pod \"dns-default-9v7ql\" (UID: \"60ba7521-2f26-41e6-a17c-98e48bf83c94\") " pod="openshift-dns/dns-default-9v7ql"
Sep 17 20:50:02.469737 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:50:02.469726    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 20:50:02.469819 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:50:02.469792    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60ba7521-2f26-41e6-a17c-98e48bf83c94-metrics-tls podName:60ba7521-2f26-41e6-a17c-98e48bf83c94 nodeName:}" failed. No retries permitted until 2026-09-17 20:50:34.469778779 +0000 UTC m=+98.815380582 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60ba7521-2f26-41e6-a17c-98e48bf83c94-metrics-tls") pod "dns-default-9v7ql" (UID: "60ba7521-2f26-41e6-a17c-98e48bf83c94") : secret "dns-default-metrics-tls" not found
Sep 17 20:50:02.470892 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:50:02.470869    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6f7a65c_e785_4e19_aed2_5c2583881c06.slice/crio-b65c9401608d0fbedd1df1408ba931ef7880f096debf620185961d0011d4c345 WatchSource:0}: Error finding container b65c9401608d0fbedd1df1408ba931ef7880f096debf620185961d0011d4c345: Status 404 returned error can't find the container with id b65c9401608d0fbedd1df1408ba931ef7880f096debf620185961d0011d4c345
Sep 17 20:50:03.443044 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:03.442999    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rb4d7" event={"ID":"c6f7a65c-e785-4e19-aed2-5c2583881c06","Type":"ContainerStarted","Data":"b65c9401608d0fbedd1df1408ba931ef7880f096debf620185961d0011d4c345"}
Sep 17 20:50:07.455645 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:07.455603    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-r7jbg" event={"ID":"082f604e-de6c-43c3-bd30-02318c0e01a4","Type":"ContainerStarted","Data":"3a7d8fa08eda950481bb9ecaeb5937ef5bc2fa97f148415678bd318ec5d009fe"}
Sep 17 20:50:07.457036 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:07.457009    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rb4d7" event={"ID":"c6f7a65c-e785-4e19-aed2-5c2583881c06","Type":"ContainerStarted","Data":"de5327206e6c02fc5f1e9259ac3bea7e4ab5fc81f2c95420f9d6ed5368665569"}
Sep 17 20:50:07.457147 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:07.457069    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:50:07.473167 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:07.473121    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-r7jbg" podStartSLOduration=66.750073418 podStartE2EDuration="1m11.473107171s" totalImagesPullingTime="4.723033753s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 20:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:50:07.472013142 +0000 UTC m=+71.817614966" watchObservedRunningTime="2026-09-17 20:50:07.473107171 +0000 UTC m=+71.818708995"
Sep 17 20:50:07.488049 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:07.488004    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rb4d7" podStartSLOduration=67.069347134 podStartE2EDuration="1m11.487994552s" totalImagesPullingTime="4.418647418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:50:07.487076965 +0000 UTC m=+71.832678790" watchObservedRunningTime="2026-09-17 20:50:07.487994552 +0000 UTC m=+71.833596372"
Sep 17 20:50:18.769062 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:18.768936    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:18.868276 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:18.868238    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:19.490043 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:19.490013    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:19.560567 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:19.560535    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:34.395858 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:34.395804    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-tls\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:50:34.396413 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:50:34.395989    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 20:50:34.396413 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:50:34.396015    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-854db97696-6sllh: secret "image-registry-tls" not found
Sep 17 20:50:34.396413 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:50:34.396092    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-tls podName:662880e7-4dd4-4556-a2db-ba6b415784aa nodeName:}" failed. No retries permitted until 2026-09-17 20:51:38.396071891 +0000 UTC m=+162.741673697 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-tls") pod "image-registry-854db97696-6sllh" (UID: "662880e7-4dd4-4556-a2db-ba6b415784aa") : secret "image-registry-tls" not found
Sep 17 20:50:34.496478 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:34.496443    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60ba7521-2f26-41e6-a17c-98e48bf83c94-metrics-tls\") pod \"dns-default-9v7ql\" (UID: \"60ba7521-2f26-41e6-a17c-98e48bf83c94\") " pod="openshift-dns/dns-default-9v7ql"
Sep 17 20:50:34.496629 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:50:34.496565    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 20:50:34.496629 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:50:34.496626    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60ba7521-2f26-41e6-a17c-98e48bf83c94-metrics-tls podName:60ba7521-2f26-41e6-a17c-98e48bf83c94 nodeName:}" failed. No retries permitted until 2026-09-17 20:51:38.496610944 +0000 UTC m=+162.842212748 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60ba7521-2f26-41e6-a17c-98e48bf83c94-metrics-tls") pod "dns-default-9v7ql" (UID: "60ba7521-2f26-41e6-a17c-98e48bf83c94") : secret "dns-default-metrics-tls" not found
Sep 17 20:50:38.462052 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:50:38.462019    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rb4d7"
Sep 17 20:51:05.903795 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:05.903738    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-metrics-certs\") pod \"network-metrics-daemon-jhhl5\" (UID: \"b4f3e778-0cec-45fe-be1b-a8789d63c1d7\") " pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:51:05.904278 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:05.903885    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 17 20:51:05.904278 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:05.903948    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-metrics-certs podName:b4f3e778-0cec-45fe-be1b-a8789d63c1d7 nodeName:}" failed. No retries permitted until 2026-09-17 20:53:07.903932219 +0000 UTC m=+252.249534026 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-metrics-certs") pod "network-metrics-daemon-jhhl5" (UID: "b4f3e778-0cec-45fe-be1b-a8789d63c1d7") : secret "metrics-daemon-secret" not found
Sep 17 20:51:15.714432 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:15.714395    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-756w5"]
Sep 17 20:51:15.716240 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:15.716225    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-ffxmr"]
Sep 17 20:51:15.716361 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:15.716343    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-756w5"
Sep 17 20:51:15.718193 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:15.718170    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-ffxmr"
Sep 17 20:51:15.721895 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:15.721875    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 17 20:51:15.722982 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:15.722961    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 17 20:51:15.723342 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:15.723326    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 17 20:51:15.724013 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:15.723999    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-tkpbs\""
Sep 17 20:51:15.724013 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:15.724004    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-m8qzf\""
Sep 17 20:51:15.735255 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:15.735236    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-756w5"]
Sep 17 20:51:15.735829 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:15.735811    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-ffxmr"]
Sep 17 20:51:15.806138 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:15.806105    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-qf2vp"]
Sep 17 20:51:15.807955 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:15.807939    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-qf2vp"
Sep 17 20:51:15.810348 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:15.810325    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 17 20:51:15.810740 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:15.810716    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-v7j2r\""
Sep 17 20:51:15.810870 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:15.810852    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 17 20:51:15.824546 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:15.824518    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-qf2vp"]
Sep 17 20:51:15.872923 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:15.872890    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/451c40ce-6a3d-44f0-b7e6-91fcdcb68c24-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-qf2vp\" (UID: \"451c40ce-6a3d-44f0-b7e6-91fcdcb68c24\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-qf2vp"
Sep 17 20:51:15.873070 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:15.872942    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwvpz\" (UniqueName: \"kubernetes.io/projected/43630f5b-7add-4f60-8fcb-425d45882a63-kube-api-access-bwvpz\") pod \"network-check-source-7cbc45fddc-ffxmr\" (UID: \"43630f5b-7add-4f60-8fcb-425d45882a63\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-ffxmr"
Sep 17 20:51:15.873070 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:15.872988    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ef83bf3-8edf-4e2d-820c-615faa739870-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-756w5\" (UID: \"2ef83bf3-8edf-4e2d-820c-615faa739870\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-756w5"
Sep 17 20:51:15.873070 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:15.873008    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79c6q\" (UniqueName: \"kubernetes.io/projected/2ef83bf3-8edf-4e2d-820c-615faa739870-kube-api-access-79c6q\") pod \"cluster-samples-operator-685d7d68b5-756w5\" (UID: \"2ef83bf3-8edf-4e2d-820c-615faa739870\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-756w5"
Sep 17 20:51:15.873070 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:15.873051    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/451c40ce-6a3d-44f0-b7e6-91fcdcb68c24-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-qf2vp\" (UID: \"451c40ce-6a3d-44f0-b7e6-91fcdcb68c24\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-qf2vp"
Sep 17 20:51:15.973468 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:15.973391    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/451c40ce-6a3d-44f0-b7e6-91fcdcb68c24-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-qf2vp\" (UID: \"451c40ce-6a3d-44f0-b7e6-91fcdcb68c24\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-qf2vp"
Sep 17 20:51:15.973468 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:15.973435    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwvpz\" (UniqueName: \"kubernetes.io/projected/43630f5b-7add-4f60-8fcb-425d45882a63-kube-api-access-bwvpz\") pod \"network-check-source-7cbc45fddc-ffxmr\" (UID: \"43630f5b-7add-4f60-8fcb-425d45882a63\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-ffxmr"
Sep 17 20:51:15.973632 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:15.973506    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ef83bf3-8edf-4e2d-820c-615faa739870-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-756w5\" (UID: \"2ef83bf3-8edf-4e2d-820c-615faa739870\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-756w5"
Sep 17 20:51:15.973632 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:15.973542    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79c6q\" (UniqueName: \"kubernetes.io/projected/2ef83bf3-8edf-4e2d-820c-615faa739870-kube-api-access-79c6q\") pod \"cluster-samples-operator-685d7d68b5-756w5\" (UID: \"2ef83bf3-8edf-4e2d-820c-615faa739870\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-756w5"
Sep 17 20:51:15.973632 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:15.973581    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/451c40ce-6a3d-44f0-b7e6-91fcdcb68c24-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-qf2vp\" (UID: \"451c40ce-6a3d-44f0-b7e6-91fcdcb68c24\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-qf2vp"
Sep 17 20:51:15.973751 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:15.973659    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 17 20:51:15.973751 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:15.973664    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 17 20:51:15.973751 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:15.973720    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ef83bf3-8edf-4e2d-820c-615faa739870-samples-operator-tls podName:2ef83bf3-8edf-4e2d-820c-615faa739870 nodeName:}" failed. No retries permitted until 2026-09-17 20:51:16.473701465 +0000 UTC m=+140.819303279 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2ef83bf3-8edf-4e2d-820c-615faa739870-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-756w5" (UID: "2ef83bf3-8edf-4e2d-820c-615faa739870") : secret "samples-operator-tls" not found
Sep 17 20:51:15.973892 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:15.973785    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/451c40ce-6a3d-44f0-b7e6-91fcdcb68c24-networking-console-plugin-cert podName:451c40ce-6a3d-44f0-b7e6-91fcdcb68c24 nodeName:}" failed. No retries permitted until 2026-09-17 20:51:16.473748261 +0000 UTC m=+140.819350064 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/451c40ce-6a3d-44f0-b7e6-91fcdcb68c24-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-qf2vp" (UID: "451c40ce-6a3d-44f0-b7e6-91fcdcb68c24") : secret "networking-console-plugin-cert" not found
Sep 17 20:51:15.974248 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:15.974230    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/451c40ce-6a3d-44f0-b7e6-91fcdcb68c24-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-qf2vp\" (UID: \"451c40ce-6a3d-44f0-b7e6-91fcdcb68c24\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-qf2vp"
Sep 17 20:51:15.985316 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:15.984994    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79c6q\" (UniqueName: \"kubernetes.io/projected/2ef83bf3-8edf-4e2d-820c-615faa739870-kube-api-access-79c6q\") pod \"cluster-samples-operator-685d7d68b5-756w5\" (UID: \"2ef83bf3-8edf-4e2d-820c-615faa739870\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-756w5"
Sep 17 20:51:15.986373 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:15.986348    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwvpz\" (UniqueName: \"kubernetes.io/projected/43630f5b-7add-4f60-8fcb-425d45882a63-kube-api-access-bwvpz\") pod \"network-check-source-7cbc45fddc-ffxmr\" (UID: \"43630f5b-7add-4f60-8fcb-425d45882a63\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-ffxmr"
Sep 17 20:51:16.018108 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:16.018068    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:16.031194 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:16.031162    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-ffxmr"
Sep 17 20:51:16.295564 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:16.295494    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-ffxmr"]
Sep 17 20:51:16.298681 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:51:16.298645    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43630f5b_7add_4f60_8fcb_425d45882a63.slice/crio-0fa847226ef46d217605f8435b6301511eeb0f6a517927a1a8132a8645ca23e5 WatchSource:0}: Error finding container 0fa847226ef46d217605f8435b6301511eeb0f6a517927a1a8132a8645ca23e5: Status 404 returned error can't find the container with id 0fa847226ef46d217605f8435b6301511eeb0f6a517927a1a8132a8645ca23e5
Sep 17 20:51:16.478005 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:16.477964    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ef83bf3-8edf-4e2d-820c-615faa739870-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-756w5\" (UID: \"2ef83bf3-8edf-4e2d-820c-615faa739870\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-756w5"
Sep 17 20:51:16.478177 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:16.478077    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/451c40ce-6a3d-44f0-b7e6-91fcdcb68c24-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-qf2vp\" (UID: \"451c40ce-6a3d-44f0-b7e6-91fcdcb68c24\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-qf2vp"
Sep 17 20:51:16.478177 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:16.478114    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 17 20:51:16.478259 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:16.478180    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ef83bf3-8edf-4e2d-820c-615faa739870-samples-operator-tls podName:2ef83bf3-8edf-4e2d-820c-615faa739870 nodeName:}" failed. No retries permitted until 2026-09-17 20:51:17.47816498 +0000 UTC m=+141.823766783 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2ef83bf3-8edf-4e2d-820c-615faa739870-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-756w5" (UID: "2ef83bf3-8edf-4e2d-820c-615faa739870") : secret "samples-operator-tls" not found
Sep 17 20:51:16.478259 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:16.478205    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 17 20:51:16.478259 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:16.478257    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/451c40ce-6a3d-44f0-b7e6-91fcdcb68c24-networking-console-plugin-cert podName:451c40ce-6a3d-44f0-b7e6-91fcdcb68c24 nodeName:}" failed. No retries permitted until 2026-09-17 20:51:17.478241449 +0000 UTC m=+141.823843263 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/451c40ce-6a3d-44f0-b7e6-91fcdcb68c24-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-qf2vp" (UID: "451c40ce-6a3d-44f0-b7e6-91fcdcb68c24") : secret "networking-console-plugin-cert" not found
Sep 17 20:51:16.623244 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:16.623155    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-ffxmr" event={"ID":"43630f5b-7add-4f60-8fcb-425d45882a63","Type":"ContainerStarted","Data":"7809a3d1f9fd05ff81e535cdd09a4327d5bcc49247d5bf4e2aacd307936486ce"}
Sep 17 20:51:16.623244 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:16.623211    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-ffxmr" event={"ID":"43630f5b-7add-4f60-8fcb-425d45882a63","Type":"ContainerStarted","Data":"0fa847226ef46d217605f8435b6301511eeb0f6a517927a1a8132a8645ca23e5"}
Sep 17 20:51:16.641653 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:16.641607    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-ffxmr" podStartSLOduration=1.6415907920000001 podStartE2EDuration="1.641590792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:51:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:51:16.641506862 +0000 UTC m=+140.987108709" watchObservedRunningTime="2026-09-17 20:51:16.641590792 +0000 UTC m=+140.987192617"
Sep 17 20:51:17.219640 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:17.219607    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:17.486050 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:17.485955    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/451c40ce-6a3d-44f0-b7e6-91fcdcb68c24-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-qf2vp\" (UID: \"451c40ce-6a3d-44f0-b7e6-91fcdcb68c24\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-qf2vp"
Sep 17 20:51:17.486050 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:17.486019    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ef83bf3-8edf-4e2d-820c-615faa739870-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-756w5\" (UID: \"2ef83bf3-8edf-4e2d-820c-615faa739870\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-756w5"
Sep 17 20:51:17.486233 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:17.486115    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 17 20:51:17.486233 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:17.486134    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 17 20:51:17.486233 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:17.486185    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/451c40ce-6a3d-44f0-b7e6-91fcdcb68c24-networking-console-plugin-cert podName:451c40ce-6a3d-44f0-b7e6-91fcdcb68c24 nodeName:}" failed. No retries permitted until 2026-09-17 20:51:19.486166886 +0000 UTC m=+143.831768691 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/451c40ce-6a3d-44f0-b7e6-91fcdcb68c24-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-qf2vp" (UID: "451c40ce-6a3d-44f0-b7e6-91fcdcb68c24") : secret "networking-console-plugin-cert" not found
Sep 17 20:51:17.486233 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:17.486201    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ef83bf3-8edf-4e2d-820c-615faa739870-samples-operator-tls podName:2ef83bf3-8edf-4e2d-820c-615faa739870 nodeName:}" failed. No retries permitted until 2026-09-17 20:51:19.486194462 +0000 UTC m=+143.831796264 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2ef83bf3-8edf-4e2d-820c-615faa739870-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-756w5" (UID: "2ef83bf3-8edf-4e2d-820c-615faa739870") : secret "samples-operator-tls" not found
Sep 17 20:51:17.960948 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:17.960914    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:18.061677 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:18.061638    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:19.457650 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:19.457616    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:19.501916 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:19.501883    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/451c40ce-6a3d-44f0-b7e6-91fcdcb68c24-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-qf2vp\" (UID: \"451c40ce-6a3d-44f0-b7e6-91fcdcb68c24\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-qf2vp"
Sep 17 20:51:19.502103 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:19.501930    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ef83bf3-8edf-4e2d-820c-615faa739870-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-756w5\" (UID: \"2ef83bf3-8edf-4e2d-820c-615faa739870\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-756w5"
Sep 17 20:51:19.502103 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:19.502034    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 17 20:51:19.502103 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:19.502056    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 17 20:51:19.502103 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:19.502089    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ef83bf3-8edf-4e2d-820c-615faa739870-samples-operator-tls podName:2ef83bf3-8edf-4e2d-820c-615faa739870 nodeName:}" failed. No retries permitted until 2026-09-17 20:51:23.502073892 +0000 UTC m=+147.847675699 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2ef83bf3-8edf-4e2d-820c-615faa739870-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-756w5" (UID: "2ef83bf3-8edf-4e2d-820c-615faa739870") : secret "samples-operator-tls" not found
Sep 17 20:51:19.502308 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:19.502118    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/451c40ce-6a3d-44f0-b7e6-91fcdcb68c24-networking-console-plugin-cert podName:451c40ce-6a3d-44f0-b7e6-91fcdcb68c24 nodeName:}" failed. No retries permitted until 2026-09-17 20:51:23.502100545 +0000 UTC m=+147.847702348 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/451c40ce-6a3d-44f0-b7e6-91fcdcb68c24-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-qf2vp" (UID: "451c40ce-6a3d-44f0-b7e6-91fcdcb68c24") : secret "networking-console-plugin-cert" not found
Sep 17 20:51:19.599356 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:19.599319    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:19.669366 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:19.669332    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:19.919263 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:19.919225    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:20.023208 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:20.023170    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:20.199766 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:20.199680    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nc246"]
Sep 17 20:51:20.202675 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:20.202654    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nc246"
Sep 17 20:51:20.206487 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:20.206460    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 17 20:51:20.207367 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:20.207344    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 17 20:51:20.207475 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:20.207339    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 17 20:51:20.207475 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:20.207341    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 17 20:51:20.207593 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:20.207351    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5hgxg\""
Sep 17 20:51:20.216751 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:20.216727    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nc246"]
Sep 17 20:51:20.234564 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:20.234533    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:20.308885 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:20.308846    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krjks\" (UniqueName: \"kubernetes.io/projected/5de2dae8-8948-48c7-b710-aa7f85b8abdd-kube-api-access-krjks\") pod \"insights-runtime-extractor-nc246\" (UID: \"5de2dae8-8948-48c7-b710-aa7f85b8abdd\") " pod="openshift-insights/insights-runtime-extractor-nc246"
Sep 17 20:51:20.309058 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:20.308899    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5de2dae8-8948-48c7-b710-aa7f85b8abdd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nc246\" (UID: \"5de2dae8-8948-48c7-b710-aa7f85b8abdd\") " pod="openshift-insights/insights-runtime-extractor-nc246"
Sep 17 20:51:20.309058 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:20.308933    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5de2dae8-8948-48c7-b710-aa7f85b8abdd-crio-socket\") pod \"insights-runtime-extractor-nc246\" (UID: \"5de2dae8-8948-48c7-b710-aa7f85b8abdd\") " pod="openshift-insights/insights-runtime-extractor-nc246"
Sep 17 20:51:20.309058 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:20.308951    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5de2dae8-8948-48c7-b710-aa7f85b8abdd-data-volume\") pod \"insights-runtime-extractor-nc246\" (UID: \"5de2dae8-8948-48c7-b710-aa7f85b8abdd\") " pod="openshift-insights/insights-runtime-extractor-nc246"
Sep 17 20:51:20.309058 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:20.308974    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5de2dae8-8948-48c7-b710-aa7f85b8abdd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nc246\" (UID: \"5de2dae8-8948-48c7-b710-aa7f85b8abdd\") " pod="openshift-insights/insights-runtime-extractor-nc246"
Sep 17 20:51:20.313323 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:20.313286    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:20.409337 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:20.409301    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krjks\" (UniqueName: \"kubernetes.io/projected/5de2dae8-8948-48c7-b710-aa7f85b8abdd-kube-api-access-krjks\") pod \"insights-runtime-extractor-nc246\" (UID: \"5de2dae8-8948-48c7-b710-aa7f85b8abdd\") " pod="openshift-insights/insights-runtime-extractor-nc246"
Sep 17 20:51:20.409519 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:20.409348    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5de2dae8-8948-48c7-b710-aa7f85b8abdd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nc246\" (UID: \"5de2dae8-8948-48c7-b710-aa7f85b8abdd\") " pod="openshift-insights/insights-runtime-extractor-nc246"
Sep 17 20:51:20.409519 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:20.409423    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5de2dae8-8948-48c7-b710-aa7f85b8abdd-crio-socket\") pod \"insights-runtime-extractor-nc246\" (UID: \"5de2dae8-8948-48c7-b710-aa7f85b8abdd\") " pod="openshift-insights/insights-runtime-extractor-nc246"
Sep 17 20:51:20.409519 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:20.409443    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 20:51:20.409519 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:20.409464    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5de2dae8-8948-48c7-b710-aa7f85b8abdd-data-volume\") pod \"insights-runtime-extractor-nc246\" (UID: \"5de2dae8-8948-48c7-b710-aa7f85b8abdd\") " pod="openshift-insights/insights-runtime-extractor-nc246"
Sep 17 20:51:20.409519 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:20.409496    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5de2dae8-8948-48c7-b710-aa7f85b8abdd-insights-runtime-extractor-tls podName:5de2dae8-8948-48c7-b710-aa7f85b8abdd nodeName:}" failed. No retries permitted until 2026-09-17 20:51:20.909481535 +0000 UTC m=+145.255083338 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5de2dae8-8948-48c7-b710-aa7f85b8abdd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nc246" (UID: "5de2dae8-8948-48c7-b710-aa7f85b8abdd") : secret "insights-runtime-extractor-tls" not found
Sep 17 20:51:20.409519 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:20.409518    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5de2dae8-8948-48c7-b710-aa7f85b8abdd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nc246\" (UID: \"5de2dae8-8948-48c7-b710-aa7f85b8abdd\") " pod="openshift-insights/insights-runtime-extractor-nc246"
Sep 17 20:51:20.409856 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:20.409608    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 20:51:20.409856 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:20.409603    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5de2dae8-8948-48c7-b710-aa7f85b8abdd-crio-socket\") pod \"insights-runtime-extractor-nc246\" (UID: \"5de2dae8-8948-48c7-b710-aa7f85b8abdd\") " pod="openshift-insights/insights-runtime-extractor-nc246"
Sep 17 20:51:20.409856 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:20.409649    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5de2dae8-8948-48c7-b710-aa7f85b8abdd-kube-rbac-proxy-cm podName:5de2dae8-8948-48c7-b710-aa7f85b8abdd nodeName:}" failed. No retries permitted until 2026-09-17 20:51:20.909639046 +0000 UTC m=+145.255240852 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5de2dae8-8948-48c7-b710-aa7f85b8abdd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nc246" (UID: "5de2dae8-8948-48c7-b710-aa7f85b8abdd") : configmap "kube-rbac-proxy" not found
Sep 17 20:51:20.409856 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:20.409826    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5de2dae8-8948-48c7-b710-aa7f85b8abdd-data-volume\") pod \"insights-runtime-extractor-nc246\" (UID: \"5de2dae8-8948-48c7-b710-aa7f85b8abdd\") " pod="openshift-insights/insights-runtime-extractor-nc246"
Sep 17 20:51:20.419132 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:20.419110    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krjks\" (UniqueName: \"kubernetes.io/projected/5de2dae8-8948-48c7-b710-aa7f85b8abdd-kube-api-access-krjks\") pod \"insights-runtime-extractor-nc246\" (UID: \"5de2dae8-8948-48c7-b710-aa7f85b8abdd\") " pod="openshift-insights/insights-runtime-extractor-nc246"
Sep 17 20:51:20.590677 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:20.590645    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:20.686174 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:20.686137    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:20.913252 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:20.913151    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5de2dae8-8948-48c7-b710-aa7f85b8abdd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nc246\" (UID: \"5de2dae8-8948-48c7-b710-aa7f85b8abdd\") " pod="openshift-insights/insights-runtime-extractor-nc246"
Sep 17 20:51:20.913252 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:20.913245    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5de2dae8-8948-48c7-b710-aa7f85b8abdd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nc246\" (UID: \"5de2dae8-8948-48c7-b710-aa7f85b8abdd\") " pod="openshift-insights/insights-runtime-extractor-nc246"
Sep 17 20:51:20.913471 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:20.913305    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 20:51:20.913471 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:20.913347    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 20:51:20.913471 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:20.913373    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5de2dae8-8948-48c7-b710-aa7f85b8abdd-insights-runtime-extractor-tls podName:5de2dae8-8948-48c7-b710-aa7f85b8abdd nodeName:}" failed. No retries permitted until 2026-09-17 20:51:21.913356674 +0000 UTC m=+146.258958478 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5de2dae8-8948-48c7-b710-aa7f85b8abdd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nc246" (UID: "5de2dae8-8948-48c7-b710-aa7f85b8abdd") : secret "insights-runtime-extractor-tls" not found
Sep 17 20:51:20.913471 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:20.913399    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5de2dae8-8948-48c7-b710-aa7f85b8abdd-kube-rbac-proxy-cm podName:5de2dae8-8948-48c7-b710-aa7f85b8abdd nodeName:}" failed. No retries permitted until 2026-09-17 20:51:21.913384064 +0000 UTC m=+146.258985866 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5de2dae8-8948-48c7-b710-aa7f85b8abdd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nc246" (UID: "5de2dae8-8948-48c7-b710-aa7f85b8abdd") : configmap "kube-rbac-proxy" not found
Sep 17 20:51:21.920365 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:21.920326    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5de2dae8-8948-48c7-b710-aa7f85b8abdd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nc246\" (UID: \"5de2dae8-8948-48c7-b710-aa7f85b8abdd\") " pod="openshift-insights/insights-runtime-extractor-nc246"
Sep 17 20:51:21.920737 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:21.920435    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5de2dae8-8948-48c7-b710-aa7f85b8abdd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nc246\" (UID: \"5de2dae8-8948-48c7-b710-aa7f85b8abdd\") " pod="openshift-insights/insights-runtime-extractor-nc246"
Sep 17 20:51:21.920737 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:21.920452    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 20:51:21.920737 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:21.920536    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5de2dae8-8948-48c7-b710-aa7f85b8abdd-kube-rbac-proxy-cm podName:5de2dae8-8948-48c7-b710-aa7f85b8abdd nodeName:}" failed. No retries permitted until 2026-09-17 20:51:23.920516908 +0000 UTC m=+148.266118728 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5de2dae8-8948-48c7-b710-aa7f85b8abdd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nc246" (UID: "5de2dae8-8948-48c7-b710-aa7f85b8abdd") : configmap "kube-rbac-proxy" not found
Sep 17 20:51:21.920737 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:21.920546    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 20:51:21.920737 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:21.920580    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5de2dae8-8948-48c7-b710-aa7f85b8abdd-insights-runtime-extractor-tls podName:5de2dae8-8948-48c7-b710-aa7f85b8abdd nodeName:}" failed. No retries permitted until 2026-09-17 20:51:23.920569712 +0000 UTC m=+148.266171514 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5de2dae8-8948-48c7-b710-aa7f85b8abdd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nc246" (UID: "5de2dae8-8948-48c7-b710-aa7f85b8abdd") : secret "insights-runtime-extractor-tls" not found
Sep 17 20:51:22.005586 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:22.005556    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:22.110766 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:22.110724    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:22.230297 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:22.230262    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:22.350928 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:22.350893    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:22.973420 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:22.973392    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-prdg7_39cff161-a45c-4052-a183-e206fd6360da/dns-node-resolver/0.log"
Sep 17 20:51:23.534563 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:23.534519    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/451c40ce-6a3d-44f0-b7e6-91fcdcb68c24-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-qf2vp\" (UID: \"451c40ce-6a3d-44f0-b7e6-91fcdcb68c24\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-qf2vp"
Sep 17 20:51:23.534744 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:23.534586    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ef83bf3-8edf-4e2d-820c-615faa739870-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-756w5\" (UID: \"2ef83bf3-8edf-4e2d-820c-615faa739870\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-756w5"
Sep 17 20:51:23.534744 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:23.534664    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 17 20:51:23.534744 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:23.534703    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 17 20:51:23.534744 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:23.534740    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/451c40ce-6a3d-44f0-b7e6-91fcdcb68c24-networking-console-plugin-cert podName:451c40ce-6a3d-44f0-b7e6-91fcdcb68c24 nodeName:}" failed. No retries permitted until 2026-09-17 20:51:31.534723894 +0000 UTC m=+155.880325702 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/451c40ce-6a3d-44f0-b7e6-91fcdcb68c24-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-qf2vp" (UID: "451c40ce-6a3d-44f0-b7e6-91fcdcb68c24") : secret "networking-console-plugin-cert" not found
Sep 17 20:51:23.534915 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:23.534773    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ef83bf3-8edf-4e2d-820c-615faa739870-samples-operator-tls podName:2ef83bf3-8edf-4e2d-820c-615faa739870 nodeName:}" failed. No retries permitted until 2026-09-17 20:51:31.534747841 +0000 UTC m=+155.880349644 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2ef83bf3-8edf-4e2d-820c-615faa739870-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-756w5" (UID: "2ef83bf3-8edf-4e2d-820c-615faa739870") : secret "samples-operator-tls" not found
Sep 17 20:51:23.773182 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:23.773158    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6h956_6bc4ca39-a164-4ff7-9c1c-fedf495ccb5c/node-ca/0.log"
Sep 17 20:51:23.794994 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:23.794920    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:23.938127 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:23.938081    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5de2dae8-8948-48c7-b710-aa7f85b8abdd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nc246\" (UID: \"5de2dae8-8948-48c7-b710-aa7f85b8abdd\") " pod="openshift-insights/insights-runtime-extractor-nc246"
Sep 17 20:51:23.938292 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:23.938188    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5de2dae8-8948-48c7-b710-aa7f85b8abdd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nc246\" (UID: \"5de2dae8-8948-48c7-b710-aa7f85b8abdd\") " pod="openshift-insights/insights-runtime-extractor-nc246"
Sep 17 20:51:23.938292 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:23.938204    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 20:51:23.938292 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:23.938274    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5de2dae8-8948-48c7-b710-aa7f85b8abdd-kube-rbac-proxy-cm podName:5de2dae8-8948-48c7-b710-aa7f85b8abdd nodeName:}" failed. No retries permitted until 2026-09-17 20:51:27.938260422 +0000 UTC m=+152.283862229 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5de2dae8-8948-48c7-b710-aa7f85b8abdd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nc246" (UID: "5de2dae8-8948-48c7-b710-aa7f85b8abdd") : configmap "kube-rbac-proxy" not found
Sep 17 20:51:23.938397 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:23.938327    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 20:51:23.938397 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:23.938390    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5de2dae8-8948-48c7-b710-aa7f85b8abdd-insights-runtime-extractor-tls podName:5de2dae8-8948-48c7-b710-aa7f85b8abdd nodeName:}" failed. No retries permitted until 2026-09-17 20:51:27.938375925 +0000 UTC m=+152.283977734 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5de2dae8-8948-48c7-b710-aa7f85b8abdd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nc246" (UID: "5de2dae8-8948-48c7-b710-aa7f85b8abdd") : secret "insights-runtime-extractor-tls" not found
Sep 17 20:51:24.573119 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:24.573088    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b8zb8_aa9e55ff-3a8d-477b-8165-ce0c5183b15f/serve-healthcheck-canary/0.log"
Sep 17 20:51:26.594185 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:26.594149    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:26.695227 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:26.695198    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:26.852363 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:26.852287    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:26.960059 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:26.960034    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:27.052130 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:27.052103    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:27.302484 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:27.302442    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:27.883829 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:27.883799    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:27.970792 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:27.970735    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5de2dae8-8948-48c7-b710-aa7f85b8abdd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nc246\" (UID: \"5de2dae8-8948-48c7-b710-aa7f85b8abdd\") " pod="openshift-insights/insights-runtime-extractor-nc246"
Sep 17 20:51:27.970944 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:27.970828    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5de2dae8-8948-48c7-b710-aa7f85b8abdd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nc246\" (UID: \"5de2dae8-8948-48c7-b710-aa7f85b8abdd\") " pod="openshift-insights/insights-runtime-extractor-nc246"
Sep 17 20:51:27.970944 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:27.970884    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 20:51:27.971007 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:27.970940    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 20:51:27.971007 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:27.970958    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5de2dae8-8948-48c7-b710-aa7f85b8abdd-insights-runtime-extractor-tls podName:5de2dae8-8948-48c7-b710-aa7f85b8abdd nodeName:}" failed. No retries permitted until 2026-09-17 20:51:35.970937157 +0000 UTC m=+160.316538981 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5de2dae8-8948-48c7-b710-aa7f85b8abdd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nc246" (UID: "5de2dae8-8948-48c7-b710-aa7f85b8abdd") : secret "insights-runtime-extractor-tls" not found
Sep 17 20:51:27.971007 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:27.970999    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5de2dae8-8948-48c7-b710-aa7f85b8abdd-kube-rbac-proxy-cm podName:5de2dae8-8948-48c7-b710-aa7f85b8abdd nodeName:}" failed. No retries permitted until 2026-09-17 20:51:35.970987445 +0000 UTC m=+160.316589253 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5de2dae8-8948-48c7-b710-aa7f85b8abdd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nc246" (UID: "5de2dae8-8948-48c7-b710-aa7f85b8abdd") : configmap "kube-rbac-proxy" not found
Sep 17 20:51:28.226647 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:28.226609    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:28.507169 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:28.507080    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:29.117480 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:29.117436    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:29.301812 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:29.301776    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:29.689496 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:29.689464    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:29.880734 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:29.880700    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:29.953884 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:29.953799    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:30.085004 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:30.084974    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:30.398581 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:30.398554    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:31.083037 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:31.082998    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:31.143746 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:31.143714    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:31.287605 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:31.287566    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:31.603785 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:31.603740    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ef83bf3-8edf-4e2d-820c-615faa739870-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-756w5\" (UID: \"2ef83bf3-8edf-4e2d-820c-615faa739870\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-756w5"
Sep 17 20:51:31.604147 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:31.603841    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/451c40ce-6a3d-44f0-b7e6-91fcdcb68c24-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-qf2vp\" (UID: \"451c40ce-6a3d-44f0-b7e6-91fcdcb68c24\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-qf2vp"
Sep 17 20:51:31.604147 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:31.603935    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 17 20:51:31.604147 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:31.603995    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/451c40ce-6a3d-44f0-b7e6-91fcdcb68c24-networking-console-plugin-cert podName:451c40ce-6a3d-44f0-b7e6-91fcdcb68c24 nodeName:}" failed. No retries permitted until 2026-09-17 20:51:47.603979243 +0000 UTC m=+171.949581046 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/451c40ce-6a3d-44f0-b7e6-91fcdcb68c24-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-qf2vp" (UID: "451c40ce-6a3d-44f0-b7e6-91fcdcb68c24") : secret "networking-console-plugin-cert" not found
Sep 17 20:51:31.606115 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:31.606088    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ef83bf3-8edf-4e2d-820c-615faa739870-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-756w5\" (UID: \"2ef83bf3-8edf-4e2d-820c-615faa739870\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-756w5"
Sep 17 20:51:31.625793 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:31.625767    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-756w5"
Sep 17 20:51:31.689948 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:31.689913    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:31.876497 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:31.876422    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:31.897570 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:31.894073    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-756w5"]
Sep 17 20:51:32.290934 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:32.290895    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:32.618612 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:32.618523    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:32.662329 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:32.662295    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-756w5" event={"ID":"2ef83bf3-8edf-4e2d-820c-615faa739870","Type":"ContainerStarted","Data":"1204f0f6d604e6b7e9636d7c83627c6d6ea8085a2f111c9d73e595f9bf17296b"}
Sep 17 20:51:32.723931 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:32.723875    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:33.282879 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:33.282844    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:33.570604 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:33.570569    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-854db97696-6sllh" podUID="662880e7-4dd4-4556-a2db-ba6b415784aa"
Sep 17 20:51:33.597749 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:33.597707    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-9v7ql" podUID="60ba7521-2f26-41e6-a17c-98e48bf83c94"
Sep 17 20:51:33.667952 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:33.667925    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9v7ql"
Sep 17 20:51:33.668352 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:33.667952    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-756w5" event={"ID":"2ef83bf3-8edf-4e2d-820c-615faa739870","Type":"ContainerStarted","Data":"62c6e96a2c52f89cf3de712512264146b9d7af1d3e1cb9953056100a516155b5"}
Sep 17 20:51:33.668352 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:33.668015    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-756w5" event={"ID":"2ef83bf3-8edf-4e2d-820c-615faa739870","Type":"ContainerStarted","Data":"2ebe9d44f92f5ad1363698bcb9322bee9bd9ba3cd874f9823c2ed455bc50ee45"}
Sep 17 20:51:33.688629 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:33.688574    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-756w5" podStartSLOduration=17.185501713 podStartE2EDuration="18.688554089s" totalImagesPullingTime="1.503052376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:51:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:51:33.687968419 +0000 UTC m=+158.033570246" watchObservedRunningTime="2026-09-17 20:51:33.688554089 +0000 UTC m=+158.034155915"
Sep 17 20:51:33.696316 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:33.696290    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:33.882243 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:33.882209    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:34.495242 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:34.495193    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:35.102985 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:35.102935    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:35.180408 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:35.180372    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-jhhl5" podUID="b4f3e778-0cec-45fe-be1b-a8789d63c1d7"
Sep 17 20:51:35.296213 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:35.296169    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:35.492823 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:35.492783    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:35.773439 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:35.773347    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:35.901038 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:35.901007    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:36.037714 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:36.037627    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5de2dae8-8948-48c7-b710-aa7f85b8abdd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nc246\" (UID: \"5de2dae8-8948-48c7-b710-aa7f85b8abdd\") " pod="openshift-insights/insights-runtime-extractor-nc246"
Sep 17 20:51:36.037946 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:36.037792    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5de2dae8-8948-48c7-b710-aa7f85b8abdd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nc246\" (UID: \"5de2dae8-8948-48c7-b710-aa7f85b8abdd\") " pod="openshift-insights/insights-runtime-extractor-nc246"
Sep 17 20:51:36.037946 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:36.037892    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 20:51:36.038063 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:51:36.037954    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5de2dae8-8948-48c7-b710-aa7f85b8abdd-insights-runtime-extractor-tls podName:5de2dae8-8948-48c7-b710-aa7f85b8abdd nodeName:}" failed. No retries permitted until 2026-09-17 20:51:52.037939418 +0000 UTC m=+176.383541220 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5de2dae8-8948-48c7-b710-aa7f85b8abdd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nc246" (UID: "5de2dae8-8948-48c7-b710-aa7f85b8abdd") : secret "insights-runtime-extractor-tls" not found
Sep 17 20:51:36.038270 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:36.038247    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5de2dae8-8948-48c7-b710-aa7f85b8abdd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nc246\" (UID: \"5de2dae8-8948-48c7-b710-aa7f85b8abdd\") " pod="openshift-insights/insights-runtime-extractor-nc246"
Sep 17 20:51:36.287819 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:36.287728    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:36.890286 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:36.890253    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:37.083334 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:37.083287    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:37.288411 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:37.288355    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:38.454291 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:38.454252    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-tls\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:51:38.456538 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:38.456517    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-tls\") pod \"image-registry-854db97696-6sllh\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") " pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:51:38.554937 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:38.554902    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60ba7521-2f26-41e6-a17c-98e48bf83c94-metrics-tls\") pod \"dns-default-9v7ql\" (UID: \"60ba7521-2f26-41e6-a17c-98e48bf83c94\") " pod="openshift-dns/dns-default-9v7ql"
Sep 17 20:51:38.557084 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:38.557060    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60ba7521-2f26-41e6-a17c-98e48bf83c94-metrics-tls\") pod \"dns-default-9v7ql\" (UID: \"60ba7521-2f26-41e6-a17c-98e48bf83c94\") " pod="openshift-dns/dns-default-9v7ql"
Sep 17 20:51:38.771680 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:38.771650    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lzdg7\""
Sep 17 20:51:38.779141 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:38.779122    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9v7ql"
Sep 17 20:51:39.047484 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:39.047409    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9v7ql"]
Sep 17 20:51:39.050552 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:51:39.050523    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60ba7521_2f26_41e6_a17c_98e48bf83c94.slice/crio-daa3684042a857edaa4277d547b6ed2a130111215b654fbd132ee1f068c51b0e WatchSource:0}: Error finding container daa3684042a857edaa4277d547b6ed2a130111215b654fbd132ee1f068c51b0e: Status 404 returned error can't find the container with id daa3684042a857edaa4277d547b6ed2a130111215b654fbd132ee1f068c51b0e
Sep 17 20:51:39.687374 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:39.687339    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9v7ql" event={"ID":"60ba7521-2f26-41e6-a17c-98e48bf83c94","Type":"ContainerStarted","Data":"daa3684042a857edaa4277d547b6ed2a130111215b654fbd132ee1f068c51b0e"}
Sep 17 20:51:40.691449 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:40.691416    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9v7ql" event={"ID":"60ba7521-2f26-41e6-a17c-98e48bf83c94","Type":"ContainerStarted","Data":"f0beeb7cc0b9dc86b690056c9c40e25efbc2320efd72d6224b31fb278367e215"}
Sep 17 20:51:41.696742 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:41.696705    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9v7ql" event={"ID":"60ba7521-2f26-41e6-a17c-98e48bf83c94","Type":"ContainerStarted","Data":"6c1c1a7840ae5779f831da1dc4c63fefad5cb52fdf08ec5fe4c14e238d9a7694"}
Sep 17 20:51:41.697093 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:41.696843    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9v7ql"
Sep 17 20:51:41.716096 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:41.716059    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9v7ql" podStartSLOduration=130.239254163 podStartE2EDuration="2m11.716046048s" totalImagesPullingTime="1.476791885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:49:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:51:41.715894611 +0000 UTC m=+166.061496436" watchObservedRunningTime="2026-09-17 20:51:41.716046048 +0000 UTC m=+166.061647872"
Sep 17 20:51:43.760818 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:43.760783    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:43.861378 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:43.861340    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:43.961143 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:43.961112    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:44.061374 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:44.061300    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:44.253541 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:44.253512    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:44.616895 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:44.616854    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:44.683701 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:44.683648    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:44.814285 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:44.814254    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:44.880444 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:44.880349    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:45.046029 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:45.045998    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:45.219160 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:45.219124    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:45.319191 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:45.319160    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:45.632691 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:45.632598    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:45.834938 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:45.834911    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:46.165660 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:46.165632    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:51:46.200118 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:46.200085    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:46.308261 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:46.308232    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:46.469547 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:46.469506    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:46.567937 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:46.567902    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:47.619114 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:47.619073    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/451c40ce-6a3d-44f0-b7e6-91fcdcb68c24-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-qf2vp\" (UID: \"451c40ce-6a3d-44f0-b7e6-91fcdcb68c24\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-qf2vp"
Sep 17 20:51:47.621658 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:47.621626    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/451c40ce-6a3d-44f0-b7e6-91fcdcb68c24-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-qf2vp\" (UID: \"451c40ce-6a3d-44f0-b7e6-91fcdcb68c24\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-qf2vp"
Sep 17 20:51:47.916797 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:47.916686    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-qf2vp"
Sep 17 20:51:48.184385 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:48.184314    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-qf2vp"]
Sep 17 20:51:48.187086 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:51:48.187064    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod451c40ce_6a3d_44f0_b7e6_91fcdcb68c24.slice/crio-27dd4798220c20dd442bfe075a0c1cd22ae42007339688436353d1f85fe7ba3d WatchSource:0}: Error finding container 27dd4798220c20dd442bfe075a0c1cd22ae42007339688436353d1f85fe7ba3d: Status 404 returned error can't find the container with id 27dd4798220c20dd442bfe075a0c1cd22ae42007339688436353d1f85fe7ba3d
Sep 17 20:51:48.721412 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:48.721380    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-qf2vp" event={"ID":"451c40ce-6a3d-44f0-b7e6-91fcdcb68c24","Type":"ContainerStarted","Data":"27dd4798220c20dd442bfe075a0c1cd22ae42007339688436353d1f85fe7ba3d"}
Sep 17 20:51:49.163989 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:49.163917    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:51:49.166858 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:49.166833    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9cn4m\""
Sep 17 20:51:49.175199 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:49.175181    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:51:49.461026 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:49.460998    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-854db97696-6sllh"]
Sep 17 20:51:49.464439 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:51:49.464413    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod662880e7_4dd4_4556_a2db_ba6b415784aa.slice/crio-3c2b4e91b03b5e7af0654ceac6edc8f2ba3f70eb845adb02637d2f0f1fcef058 WatchSource:0}: Error finding container 3c2b4e91b03b5e7af0654ceac6edc8f2ba3f70eb845adb02637d2f0f1fcef058: Status 404 returned error can't find the container with id 3c2b4e91b03b5e7af0654ceac6edc8f2ba3f70eb845adb02637d2f0f1fcef058
Sep 17 20:51:49.726215 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:49.726077    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-854db97696-6sllh" event={"ID":"662880e7-4dd4-4556-a2db-ba6b415784aa","Type":"ContainerStarted","Data":"8b263aa8e82f74cc129816aecdec7a49f5333b568be8a5085ce417062333badb"}
Sep 17 20:51:49.726215 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:49.726132    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-854db97696-6sllh" event={"ID":"662880e7-4dd4-4556-a2db-ba6b415784aa","Type":"ContainerStarted","Data":"3c2b4e91b03b5e7af0654ceac6edc8f2ba3f70eb845adb02637d2f0f1fcef058"}
Sep 17 20:51:49.726215 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:49.726163    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:51:49.727849 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:49.727808    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-qf2vp" event={"ID":"451c40ce-6a3d-44f0-b7e6-91fcdcb68c24","Type":"ContainerStarted","Data":"c7463c4b913004f2ade8dff4bec209bde4666a3e1aae2420aff79920aaa8dbbd"}
Sep 17 20:51:49.750195 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:49.750134    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-854db97696-6sllh" podStartSLOduration=173.750106775 podStartE2EDuration="2m53.750106775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:48:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:51:49.749960578 +0000 UTC m=+174.095562422" watchObservedRunningTime="2026-09-17 20:51:49.750106775 +0000 UTC m=+174.095708605"
Sep 17 20:51:49.767159 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:49.767111    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6446c8d75d-qf2vp" podStartSLOduration=33.72103609 podStartE2EDuration="34.767095798s" totalImagesPullingTime="1.046059708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:51:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:51:49.766545884 +0000 UTC m=+174.112147713" watchObservedRunningTime="2026-09-17 20:51:49.767095798 +0000 UTC m=+174.112697623"
Sep 17 20:51:49.859749 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:49.859718    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:49.960343 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:49.960311    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:50.061130 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:50.061040    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:50.283335 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:50.283305    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:50.375223 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:50.375130    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:50.460682 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:50.460649    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-cnr22"]
Sep 17 20:51:50.461156 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:50.461132    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:50.465524 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:50.465507    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-cnr22"
Sep 17 20:51:50.469186 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:50.469167    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 17 20:51:50.469264 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:50.469185    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-9cfl4\""
Sep 17 20:51:50.478590 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:50.478568    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-cnr22"]
Sep 17 20:51:50.543148 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:50.543098    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a5566651-8573-441b-8596-4fb8f1960f8f-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-cnr22\" (UID: \"a5566651-8573-441b-8596-4fb8f1960f8f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-cnr22"
Sep 17 20:51:50.556221 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:50.556188    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:50.644390 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:50.644315    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a5566651-8573-441b-8596-4fb8f1960f8f-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-cnr22\" (UID: \"a5566651-8573-441b-8596-4fb8f1960f8f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-cnr22"
Sep 17 20:51:50.646731 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:50.646707    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a5566651-8573-441b-8596-4fb8f1960f8f-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-cnr22\" (UID: \"a5566651-8573-441b-8596-4fb8f1960f8f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-cnr22"
Sep 17 20:51:50.774039 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:50.774009    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-cnr22"
Sep 17 20:51:51.044349 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:51.044318    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-cnr22"]
Sep 17 20:51:51.047317 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:51:51.047286    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5566651_8573_441b_8596_4fb8f1960f8f.slice/crio-599b7f0c045ab41cff269be20c38f1b9efb09b7514f1cbf1a483ac56fa908e2c WatchSource:0}: Error finding container 599b7f0c045ab41cff269be20c38f1b9efb09b7514f1cbf1a483ac56fa908e2c: Status 404 returned error can't find the container with id 599b7f0c045ab41cff269be20c38f1b9efb09b7514f1cbf1a483ac56fa908e2c
Sep 17 20:51:51.701198 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:51.701168    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9v7ql"
Sep 17 20:51:51.736365 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:51.736329    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-cnr22" event={"ID":"a5566651-8573-441b-8596-4fb8f1960f8f","Type":"ContainerStarted","Data":"599b7f0c045ab41cff269be20c38f1b9efb09b7514f1cbf1a483ac56fa908e2c"}
Sep 17 20:51:52.055204 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:52.055171    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5de2dae8-8948-48c7-b710-aa7f85b8abdd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nc246\" (UID: \"5de2dae8-8948-48c7-b710-aa7f85b8abdd\") " pod="openshift-insights/insights-runtime-extractor-nc246"
Sep 17 20:51:52.057359 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:52.057336    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5de2dae8-8948-48c7-b710-aa7f85b8abdd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nc246\" (UID: \"5de2dae8-8948-48c7-b710-aa7f85b8abdd\") " pod="openshift-insights/insights-runtime-extractor-nc246"
Sep 17 20:51:52.315228 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:52.315140    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nc246"
Sep 17 20:51:52.453773 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:52.453723    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:52.568431 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:52.568353    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:52.593160 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:52.593127    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nc246"]
Sep 17 20:51:52.596310 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:51:52.596284    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5de2dae8_8948_48c7_b710_aa7f85b8abdd.slice/crio-1780d77224d2b849a03c86a7f5ddf31870bf381a4173d21a5de4b5242f1950f9 WatchSource:0}: Error finding container 1780d77224d2b849a03c86a7f5ddf31870bf381a4173d21a5de4b5242f1950f9: Status 404 returned error can't find the container with id 1780d77224d2b849a03c86a7f5ddf31870bf381a4173d21a5de4b5242f1950f9
Sep 17 20:51:52.741447 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:52.741412    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nc246" event={"ID":"5de2dae8-8948-48c7-b710-aa7f85b8abdd","Type":"ContainerStarted","Data":"359760cac7c388bc83032322378e8e1515c7cdcc98dc8ce81f677968845a09ff"}
Sep 17 20:51:52.741603 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:52.741475    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nc246" event={"ID":"5de2dae8-8948-48c7-b710-aa7f85b8abdd","Type":"ContainerStarted","Data":"1780d77224d2b849a03c86a7f5ddf31870bf381a4173d21a5de4b5242f1950f9"}
Sep 17 20:51:52.742954 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:52.742929    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-cnr22" event={"ID":"a5566651-8573-441b-8596-4fb8f1960f8f","Type":"ContainerStarted","Data":"6ecf1a6b22c5721ef10a0744ecd4c9a4635227faa43f1e9a221e22431dd1087a"}
Sep 17 20:51:52.743123 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:52.743108    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-cnr22"
Sep 17 20:51:52.747587 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:52.747567    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-cnr22"
Sep 17 20:51:52.766209 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:52.766167    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-cnr22" podStartSLOduration=1.840494492 podStartE2EDuration="2.766152531s" totalImagesPullingTime="925.658039ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:51:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:51:52.765877149 +0000 UTC m=+177.111478975" watchObservedRunningTime="2026-09-17 20:51:52.766152531 +0000 UTC m=+177.111754355"
Sep 17 20:51:53.075454 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.075423    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b4f78d9c4-crkjg"]
Sep 17 20:51:53.078559 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.078540    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b4f78d9c4-crkjg"
Sep 17 20:51:53.081185 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.081162    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 17 20:51:53.081305 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.081232    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 17 20:51:53.082324 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.082304    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 17 20:51:53.082425 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.082304    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 17 20:51:53.082425 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.082377    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 17 20:51:53.082515 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.082441    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 17 20:51:53.082515 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.082450    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-7mwmf\""
Sep 17 20:51:53.082515 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.082507    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 17 20:51:53.094585 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.094566    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b4f78d9c4-crkjg"]
Sep 17 20:51:53.164917 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.164877    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8e8fb726-7627-4b4a-820f-af3caa14b16d-console-serving-cert\") pod \"console-7b4f78d9c4-crkjg\" (UID: \"8e8fb726-7627-4b4a-820f-af3caa14b16d\") " pod="openshift-console/console-7b4f78d9c4-crkjg"
Sep 17 20:51:53.164917 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.164916    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8e8fb726-7627-4b4a-820f-af3caa14b16d-console-oauth-config\") pod \"console-7b4f78d9c4-crkjg\" (UID: \"8e8fb726-7627-4b4a-820f-af3caa14b16d\") " pod="openshift-console/console-7b4f78d9c4-crkjg"
Sep 17 20:51:53.165086 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.164961    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8e8fb726-7627-4b4a-820f-af3caa14b16d-service-ca\") pod \"console-7b4f78d9c4-crkjg\" (UID: \"8e8fb726-7627-4b4a-820f-af3caa14b16d\") " pod="openshift-console/console-7b4f78d9c4-crkjg"
Sep 17 20:51:53.165086 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.165000    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8e8fb726-7627-4b4a-820f-af3caa14b16d-console-config\") pod \"console-7b4f78d9c4-crkjg\" (UID: \"8e8fb726-7627-4b4a-820f-af3caa14b16d\") " pod="openshift-console/console-7b4f78d9c4-crkjg"
Sep 17 20:51:53.165086 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.165051    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8e8fb726-7627-4b4a-820f-af3caa14b16d-oauth-serving-cert\") pod \"console-7b4f78d9c4-crkjg\" (UID: \"8e8fb726-7627-4b4a-820f-af3caa14b16d\") " pod="openshift-console/console-7b4f78d9c4-crkjg"
Sep 17 20:51:53.165212 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.165106    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl8tv\" (UniqueName: \"kubernetes.io/projected/8e8fb726-7627-4b4a-820f-af3caa14b16d-kube-api-access-tl8tv\") pod \"console-7b4f78d9c4-crkjg\" (UID: \"8e8fb726-7627-4b4a-820f-af3caa14b16d\") " pod="openshift-console/console-7b4f78d9c4-crkjg"
Sep 17 20:51:53.265631 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.265597    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8e8fb726-7627-4b4a-820f-af3caa14b16d-console-config\") pod \"console-7b4f78d9c4-crkjg\" (UID: \"8e8fb726-7627-4b4a-820f-af3caa14b16d\") " pod="openshift-console/console-7b4f78d9c4-crkjg"
Sep 17 20:51:53.265631 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.265637    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8e8fb726-7627-4b4a-820f-af3caa14b16d-oauth-serving-cert\") pod \"console-7b4f78d9c4-crkjg\" (UID: \"8e8fb726-7627-4b4a-820f-af3caa14b16d\") " pod="openshift-console/console-7b4f78d9c4-crkjg"
Sep 17 20:51:53.265891 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.265670    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl8tv\" (UniqueName: \"kubernetes.io/projected/8e8fb726-7627-4b4a-820f-af3caa14b16d-kube-api-access-tl8tv\") pod \"console-7b4f78d9c4-crkjg\" (UID: \"8e8fb726-7627-4b4a-820f-af3caa14b16d\") " pod="openshift-console/console-7b4f78d9c4-crkjg"
Sep 17 20:51:53.265891 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.265717    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8e8fb726-7627-4b4a-820f-af3caa14b16d-console-serving-cert\") pod \"console-7b4f78d9c4-crkjg\" (UID: \"8e8fb726-7627-4b4a-820f-af3caa14b16d\") " pod="openshift-console/console-7b4f78d9c4-crkjg"
Sep 17 20:51:53.265891 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.265732    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8e8fb726-7627-4b4a-820f-af3caa14b16d-console-oauth-config\") pod \"console-7b4f78d9c4-crkjg\" (UID: \"8e8fb726-7627-4b4a-820f-af3caa14b16d\") " pod="openshift-console/console-7b4f78d9c4-crkjg"
Sep 17 20:51:53.265891 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.265770    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8e8fb726-7627-4b4a-820f-af3caa14b16d-service-ca\") pod \"console-7b4f78d9c4-crkjg\" (UID: \"8e8fb726-7627-4b4a-820f-af3caa14b16d\") " pod="openshift-console/console-7b4f78d9c4-crkjg"
Sep 17 20:51:53.266426 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.266399    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8e8fb726-7627-4b4a-820f-af3caa14b16d-oauth-serving-cert\") pod \"console-7b4f78d9c4-crkjg\" (UID: \"8e8fb726-7627-4b4a-820f-af3caa14b16d\") " pod="openshift-console/console-7b4f78d9c4-crkjg"
Sep 17 20:51:53.266558 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.266405    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8e8fb726-7627-4b4a-820f-af3caa14b16d-service-ca\") pod \"console-7b4f78d9c4-crkjg\" (UID: \"8e8fb726-7627-4b4a-820f-af3caa14b16d\") " pod="openshift-console/console-7b4f78d9c4-crkjg"
Sep 17 20:51:53.266558 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.266444    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8e8fb726-7627-4b4a-820f-af3caa14b16d-console-config\") pod \"console-7b4f78d9c4-crkjg\" (UID: \"8e8fb726-7627-4b4a-820f-af3caa14b16d\") " pod="openshift-console/console-7b4f78d9c4-crkjg"
Sep 17 20:51:53.268007 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.267988    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8e8fb726-7627-4b4a-820f-af3caa14b16d-console-serving-cert\") pod \"console-7b4f78d9c4-crkjg\" (UID: \"8e8fb726-7627-4b4a-820f-af3caa14b16d\") " pod="openshift-console/console-7b4f78d9c4-crkjg"
Sep 17 20:51:53.268165 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.268150    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8e8fb726-7627-4b4a-820f-af3caa14b16d-console-oauth-config\") pod \"console-7b4f78d9c4-crkjg\" (UID: \"8e8fb726-7627-4b4a-820f-af3caa14b16d\") " pod="openshift-console/console-7b4f78d9c4-crkjg"
Sep 17 20:51:53.279800 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.279781    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl8tv\" (UniqueName: \"kubernetes.io/projected/8e8fb726-7627-4b4a-820f-af3caa14b16d-kube-api-access-tl8tv\") pod \"console-7b4f78d9c4-crkjg\" (UID: \"8e8fb726-7627-4b4a-820f-af3caa14b16d\") " pod="openshift-console/console-7b4f78d9c4-crkjg"
Sep 17 20:51:53.387855 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.387832    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b4f78d9c4-crkjg"
Sep 17 20:51:53.573672 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.573642    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:53.645483 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.645414    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-fvqtk"]
Sep 17 20:51:53.649430 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.649410    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fvqtk"
Sep 17 20:51:53.654973 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.654950    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 17 20:51:53.654973 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.654967    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 17 20:51:53.655154 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.654950    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 17 20:51:53.655244 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.655218    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 17 20:51:53.655313 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.655237    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 17 20:51:53.655444 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.655398    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-nr24r\""
Sep 17 20:51:53.667935 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.667911    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-fvqtk"]
Sep 17 20:51:53.693020 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.692947    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b4f78d9c4-crkjg"]
Sep 17 20:51:53.696563 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:51:53.696528    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e8fb726_7627_4b4a_820f_af3caa14b16d.slice/crio-ebd4ff569a6caee0ddf46e500e28e0782adf22b5923ba1ee613ccaeda2fcb3da WatchSource:0}: Error finding container ebd4ff569a6caee0ddf46e500e28e0782adf22b5923ba1ee613ccaeda2fcb3da: Status 404 returned error can't find the container with id ebd4ff569a6caee0ddf46e500e28e0782adf22b5923ba1ee613ccaeda2fcb3da
Sep 17 20:51:53.715429 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.715391    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:53.748323 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.748288    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nc246" event={"ID":"5de2dae8-8948-48c7-b710-aa7f85b8abdd","Type":"ContainerStarted","Data":"660a0341384b2630a62e1b145b136f4da16060498eec1a11e8afef7fff5c4325"}
Sep 17 20:51:53.749741 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.749719    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b4f78d9c4-crkjg" event={"ID":"8e8fb726-7627-4b4a-820f-af3caa14b16d","Type":"ContainerStarted","Data":"ebd4ff569a6caee0ddf46e500e28e0782adf22b5923ba1ee613ccaeda2fcb3da"}
Sep 17 20:51:53.769303 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.769279    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5f0637b4-c50b-4441-9dbf-774519510c83-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-fvqtk\" (UID: \"5f0637b4-c50b-4441-9dbf-774519510c83\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fvqtk"
Sep 17 20:51:53.769406 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.769321    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5f0637b4-c50b-4441-9dbf-774519510c83-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-fvqtk\" (UID: \"5f0637b4-c50b-4441-9dbf-774519510c83\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fvqtk"
Sep 17 20:51:53.769406 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.769364    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f0637b4-c50b-4441-9dbf-774519510c83-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-fvqtk\" (UID: \"5f0637b4-c50b-4441-9dbf-774519510c83\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fvqtk"
Sep 17 20:51:53.769504 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.769422    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vk64\" (UniqueName: \"kubernetes.io/projected/5f0637b4-c50b-4441-9dbf-774519510c83-kube-api-access-9vk64\") pod \"prometheus-operator-7bd5d56d6-fvqtk\" (UID: \"5f0637b4-c50b-4441-9dbf-774519510c83\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fvqtk"
Sep 17 20:51:53.869333 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.869289    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:53.870016 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.869991    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f0637b4-c50b-4441-9dbf-774519510c83-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-fvqtk\" (UID: \"5f0637b4-c50b-4441-9dbf-774519510c83\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fvqtk"
Sep 17 20:51:53.870143 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.870046    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vk64\" (UniqueName: \"kubernetes.io/projected/5f0637b4-c50b-4441-9dbf-774519510c83-kube-api-access-9vk64\") pod \"prometheus-operator-7bd5d56d6-fvqtk\" (UID: \"5f0637b4-c50b-4441-9dbf-774519510c83\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fvqtk"
Sep 17 20:51:53.870143 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.870076    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5f0637b4-c50b-4441-9dbf-774519510c83-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-fvqtk\" (UID: \"5f0637b4-c50b-4441-9dbf-774519510c83\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fvqtk"
Sep 17 20:51:53.870143 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.870106    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5f0637b4-c50b-4441-9dbf-774519510c83-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-fvqtk\" (UID: \"5f0637b4-c50b-4441-9dbf-774519510c83\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fvqtk"
Sep 17 20:51:53.870823 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.870785    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f0637b4-c50b-4441-9dbf-774519510c83-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-fvqtk\" (UID: \"5f0637b4-c50b-4441-9dbf-774519510c83\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fvqtk"
Sep 17 20:51:53.873077 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.873058    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5f0637b4-c50b-4441-9dbf-774519510c83-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-fvqtk\" (UID: \"5f0637b4-c50b-4441-9dbf-774519510c83\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fvqtk"
Sep 17 20:51:53.873169 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.873138    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5f0637b4-c50b-4441-9dbf-774519510c83-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-fvqtk\" (UID: \"5f0637b4-c50b-4441-9dbf-774519510c83\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fvqtk"
Sep 17 20:51:53.880706 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.880684    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vk64\" (UniqueName: \"kubernetes.io/projected/5f0637b4-c50b-4441-9dbf-774519510c83-kube-api-access-9vk64\") pod \"prometheus-operator-7bd5d56d6-fvqtk\" (UID: \"5f0637b4-c50b-4441-9dbf-774519510c83\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fvqtk"
Sep 17 20:51:53.958365 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:53.958334    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fvqtk"
Sep 17 20:51:54.248308 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:54.248233    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-fvqtk"]
Sep 17 20:51:54.463073 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:51:54.463042    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f0637b4_c50b_4441_9dbf_774519510c83.slice/crio-f9386123ee1d87b2510c6921c101aac4a236b9b4e06bf2867def1fa24fb32316 WatchSource:0}: Error finding container f9386123ee1d87b2510c6921c101aac4a236b9b4e06bf2867def1fa24fb32316: Status 404 returned error can't find the container with id f9386123ee1d87b2510c6921c101aac4a236b9b4e06bf2867def1fa24fb32316
Sep 17 20:51:54.756074 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:54.755987    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fvqtk" event={"ID":"5f0637b4-c50b-4441-9dbf-774519510c83","Type":"ContainerStarted","Data":"f9386123ee1d87b2510c6921c101aac4a236b9b4e06bf2867def1fa24fb32316"}
Sep 17 20:51:54.759837 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:54.759806    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nc246" event={"ID":"5de2dae8-8948-48c7-b710-aa7f85b8abdd","Type":"ContainerStarted","Data":"8e9ec099430bf2910904ec5bdfff1edaf722d7357a951c2a7f260d7beb1fe4dd"}
Sep 17 20:51:54.786293 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:54.786233    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nc246" podStartSLOduration=32.994492968 podStartE2EDuration="34.78621076s" totalImagesPullingTime="1.791717792s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 20:51:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:51:54.78563575 +0000 UTC m=+179.131237603" watchObservedRunningTime="2026-09-17 20:51:54.78621076 +0000 UTC m=+179.131812590"
Sep 17 20:51:56.832023 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:56.831980    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:56.934727 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:56.934637    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:57.508947 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:57.508907    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:57.607072 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:57.607035    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:57.672662 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:57.672614    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:58.624181 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:58.624086    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:58.774830 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:58.774792    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b4f78d9c4-crkjg" event={"ID":"8e8fb726-7627-4b4a-820f-af3caa14b16d","Type":"ContainerStarted","Data":"84faac994db76cd91dfd5d7fcf38550d8f007c0d9648acbd6be96af7af9c5d69"}
Sep 17 20:51:58.776515 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:58.776489    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fvqtk" event={"ID":"5f0637b4-c50b-4441-9dbf-774519510c83","Type":"ContainerStarted","Data":"408d90a400ee6fe4e6c9cf3012085294309dba41ac982743784204a694b36140"}
Sep 17 20:51:58.776632 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:58.776532    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fvqtk" event={"ID":"5f0637b4-c50b-4441-9dbf-774519510c83","Type":"ContainerStarted","Data":"9f59beac5d8102179e676d2132068cf33985cc62277d2d3e084d449d67fa5091"}
Sep 17 20:51:58.794962 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:58.794920    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b4f78d9c4-crkjg" podStartSLOduration=1.094851768 podStartE2EDuration="5.794907785s" totalImagesPullingTime="4.700056017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:51:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:51:58.794038543 +0000 UTC m=+183.139640381" watchObservedRunningTime="2026-09-17 20:51:58.794907785 +0000 UTC m=+183.140509610"
Sep 17 20:51:58.812191 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:58.812153    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fvqtk" podStartSLOduration=1.9373550929999999 podStartE2EDuration="5.812137564s" totalImagesPullingTime="3.874782471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:51:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:51:58.811178725 +0000 UTC m=+183.156780574" watchObservedRunningTime="2026-09-17 20:51:58.812137564 +0000 UTC m=+183.157739388"
Sep 17 20:51:59.652369 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:59.652334    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:59.716586 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:51:59.716549    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:00.798000 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:00.797954    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:00.910644 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:00.910601    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:01.007422 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.007394    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-c2mkj"]
Sep 17 20:52:01.008136 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.008109    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:01.011095 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.011080    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-c2mkj"
Sep 17 20:52:01.013972 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.013951    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-gc2nm\""
Sep 17 20:52:01.014104 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.014085    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 17 20:52:01.014157 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.014092    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 17 20:52:01.024425 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.024407    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-c2mkj"]
Sep 17 20:52:01.088588 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.088516    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ddkmj"]
Sep 17 20:52:01.091947 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.091928    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ddkmj"
Sep 17 20:52:01.096808 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.096785    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 17 20:52:01.097464 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.097444    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 17 20:52:01.097654 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.097637    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 17 20:52:01.105545 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.105527    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jg6qn\""
Sep 17 20:52:01.130188 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.130165    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2272fdbe-f841-4620-8d45-531c5382ce08-node-exporter-accelerators-collector-config\") pod \"node-exporter-ddkmj\" (UID: \"2272fdbe-f841-4620-8d45-531c5382ce08\") " pod="openshift-monitoring/node-exporter-ddkmj"
Sep 17 20:52:01.130270 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.130198    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c393e135-8471-4c74-bdd2-6f4fe2da9e26-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-c2mkj\" (UID: \"c393e135-8471-4c74-bdd2-6f4fe2da9e26\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-c2mkj"
Sep 17 20:52:01.130270 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.130223    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2272fdbe-f841-4620-8d45-531c5382ce08-sys\") pod \"node-exporter-ddkmj\" (UID: \"2272fdbe-f841-4620-8d45-531c5382ce08\") " pod="openshift-monitoring/node-exporter-ddkmj"
Sep 17 20:52:01.130392 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.130309    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2272fdbe-f841-4620-8d45-531c5382ce08-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ddkmj\" (UID: \"2272fdbe-f841-4620-8d45-531c5382ce08\") " pod="openshift-monitoring/node-exporter-ddkmj"
Sep 17 20:52:01.130392 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.130342    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2272fdbe-f841-4620-8d45-531c5382ce08-node-exporter-tls\") pod \"node-exporter-ddkmj\" (UID: \"2272fdbe-f841-4620-8d45-531c5382ce08\") " pod="openshift-monitoring/node-exporter-ddkmj"
Sep 17 20:52:01.130392 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.130375    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpxvk\" (UniqueName: \"kubernetes.io/projected/2272fdbe-f841-4620-8d45-531c5382ce08-kube-api-access-vpxvk\") pod \"node-exporter-ddkmj\" (UID: \"2272fdbe-f841-4620-8d45-531c5382ce08\") " pod="openshift-monitoring/node-exporter-ddkmj"
Sep 17 20:52:01.130510 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.130408    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fp27z\" (UniqueName: \"kubernetes.io/projected/c393e135-8471-4c74-bdd2-6f4fe2da9e26-kube-api-access-fp27z\") pod \"openshift-state-metrics-86c5bc79d6-c2mkj\" (UID: \"c393e135-8471-4c74-bdd2-6f4fe2da9e26\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-c2mkj"
Sep 17 20:52:01.130510 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.130436    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c393e135-8471-4c74-bdd2-6f4fe2da9e26-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-c2mkj\" (UID: \"c393e135-8471-4c74-bdd2-6f4fe2da9e26\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-c2mkj"
Sep 17 20:52:01.130510 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.130489    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2272fdbe-f841-4620-8d45-531c5382ce08-node-exporter-wtmp\") pod \"node-exporter-ddkmj\" (UID: \"2272fdbe-f841-4620-8d45-531c5382ce08\") " pod="openshift-monitoring/node-exporter-ddkmj"
Sep 17 20:52:01.130614 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.130511    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2272fdbe-f841-4620-8d45-531c5382ce08-root\") pod \"node-exporter-ddkmj\" (UID: \"2272fdbe-f841-4620-8d45-531c5382ce08\") " pod="openshift-monitoring/node-exporter-ddkmj"
Sep 17 20:52:01.130614 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.130528    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2272fdbe-f841-4620-8d45-531c5382ce08-metrics-client-ca\") pod \"node-exporter-ddkmj\" (UID: \"2272fdbe-f841-4620-8d45-531c5382ce08\") " pod="openshift-monitoring/node-exporter-ddkmj"
Sep 17 20:52:01.130614 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.130558    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2272fdbe-f841-4620-8d45-531c5382ce08-node-exporter-textfile\") pod \"node-exporter-ddkmj\" (UID: \"2272fdbe-f841-4620-8d45-531c5382ce08\") " pod="openshift-monitoring/node-exporter-ddkmj"
Sep 17 20:52:01.130614 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.130576    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c393e135-8471-4c74-bdd2-6f4fe2da9e26-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-c2mkj\" (UID: \"c393e135-8471-4c74-bdd2-6f4fe2da9e26\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-c2mkj"
Sep 17 20:52:01.137244 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.137224    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:01.231380 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.231351    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2272fdbe-f841-4620-8d45-531c5382ce08-node-exporter-accelerators-collector-config\") pod \"node-exporter-ddkmj\" (UID: \"2272fdbe-f841-4620-8d45-531c5382ce08\") " pod="openshift-monitoring/node-exporter-ddkmj"
Sep 17 20:52:01.231565 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.231389    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c393e135-8471-4c74-bdd2-6f4fe2da9e26-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-c2mkj\" (UID: \"c393e135-8471-4c74-bdd2-6f4fe2da9e26\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-c2mkj"
Sep 17 20:52:01.231565 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.231409    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2272fdbe-f841-4620-8d45-531c5382ce08-sys\") pod \"node-exporter-ddkmj\" (UID: \"2272fdbe-f841-4620-8d45-531c5382ce08\") " pod="openshift-monitoring/node-exporter-ddkmj"
Sep 17 20:52:01.231565 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.231427    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2272fdbe-f841-4620-8d45-531c5382ce08-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ddkmj\" (UID: \"2272fdbe-f841-4620-8d45-531c5382ce08\") " pod="openshift-monitoring/node-exporter-ddkmj"
Sep 17 20:52:01.231565 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.231443    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2272fdbe-f841-4620-8d45-531c5382ce08-node-exporter-tls\") pod \"node-exporter-ddkmj\" (UID: \"2272fdbe-f841-4620-8d45-531c5382ce08\") " pod="openshift-monitoring/node-exporter-ddkmj"
Sep 17 20:52:01.231565 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.231459    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpxvk\" (UniqueName: \"kubernetes.io/projected/2272fdbe-f841-4620-8d45-531c5382ce08-kube-api-access-vpxvk\") pod \"node-exporter-ddkmj\" (UID: \"2272fdbe-f841-4620-8d45-531c5382ce08\") " pod="openshift-monitoring/node-exporter-ddkmj"
Sep 17 20:52:01.231565 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.231481    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fp27z\" (UniqueName: \"kubernetes.io/projected/c393e135-8471-4c74-bdd2-6f4fe2da9e26-kube-api-access-fp27z\") pod \"openshift-state-metrics-86c5bc79d6-c2mkj\" (UID: \"c393e135-8471-4c74-bdd2-6f4fe2da9e26\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-c2mkj"
Sep 17 20:52:01.231565 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.231493    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2272fdbe-f841-4620-8d45-531c5382ce08-sys\") pod \"node-exporter-ddkmj\" (UID: \"2272fdbe-f841-4620-8d45-531c5382ce08\") " pod="openshift-monitoring/node-exporter-ddkmj"
Sep 17 20:52:01.231565 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.231506    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c393e135-8471-4c74-bdd2-6f4fe2da9e26-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-c2mkj\" (UID: \"c393e135-8471-4c74-bdd2-6f4fe2da9e26\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-c2mkj"
Sep 17 20:52:01.231565 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:52:01.231538    2567 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 17 20:52:01.231565 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.231559    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2272fdbe-f841-4620-8d45-531c5382ce08-node-exporter-wtmp\") pod \"node-exporter-ddkmj\" (UID: \"2272fdbe-f841-4620-8d45-531c5382ce08\") " pod="openshift-monitoring/node-exporter-ddkmj"
Sep 17 20:52:01.232156 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.231587    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2272fdbe-f841-4620-8d45-531c5382ce08-root\") pod \"node-exporter-ddkmj\" (UID: \"2272fdbe-f841-4620-8d45-531c5382ce08\") " pod="openshift-monitoring/node-exporter-ddkmj"
Sep 17 20:52:01.232156 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:52:01.231612    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2272fdbe-f841-4620-8d45-531c5382ce08-node-exporter-tls podName:2272fdbe-f841-4620-8d45-531c5382ce08 nodeName:}" failed. No retries permitted until 2026-09-17 20:52:01.731592227 +0000 UTC m=+186.077194030 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/2272fdbe-f841-4620-8d45-531c5382ce08-node-exporter-tls") pod "node-exporter-ddkmj" (UID: "2272fdbe-f841-4620-8d45-531c5382ce08") : secret "node-exporter-tls" not found
Sep 17 20:52:01.232156 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.231629    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2272fdbe-f841-4620-8d45-531c5382ce08-root\") pod \"node-exporter-ddkmj\" (UID: \"2272fdbe-f841-4620-8d45-531c5382ce08\") " pod="openshift-monitoring/node-exporter-ddkmj"
Sep 17 20:52:01.232156 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.231651    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2272fdbe-f841-4620-8d45-531c5382ce08-metrics-client-ca\") pod \"node-exporter-ddkmj\" (UID: \"2272fdbe-f841-4620-8d45-531c5382ce08\") " pod="openshift-monitoring/node-exporter-ddkmj"
Sep 17 20:52:01.232156 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.231683    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2272fdbe-f841-4620-8d45-531c5382ce08-node-exporter-textfile\") pod \"node-exporter-ddkmj\" (UID: \"2272fdbe-f841-4620-8d45-531c5382ce08\") " pod="openshift-monitoring/node-exporter-ddkmj"
Sep 17 20:52:01.232156 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.231715    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c393e135-8471-4c74-bdd2-6f4fe2da9e26-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-c2mkj\" (UID: \"c393e135-8471-4c74-bdd2-6f4fe2da9e26\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-c2mkj"
Sep 17 20:52:01.232156 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.231746    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2272fdbe-f841-4620-8d45-531c5382ce08-node-exporter-wtmp\") pod \"node-exporter-ddkmj\" (UID: \"2272fdbe-f841-4620-8d45-531c5382ce08\") " pod="openshift-monitoring/node-exporter-ddkmj"
Sep 17 20:52:01.232492 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.232343    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c393e135-8471-4c74-bdd2-6f4fe2da9e26-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-c2mkj\" (UID: \"c393e135-8471-4c74-bdd2-6f4fe2da9e26\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-c2mkj"
Sep 17 20:52:01.232492 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.232370    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2272fdbe-f841-4620-8d45-531c5382ce08-metrics-client-ca\") pod \"node-exporter-ddkmj\" (UID: \"2272fdbe-f841-4620-8d45-531c5382ce08\") " pod="openshift-monitoring/node-exporter-ddkmj"
Sep 17 20:52:01.232492 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.232412    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2272fdbe-f841-4620-8d45-531c5382ce08-node-exporter-accelerators-collector-config\") pod \"node-exporter-ddkmj\" (UID: \"2272fdbe-f841-4620-8d45-531c5382ce08\") " pod="openshift-monitoring/node-exporter-ddkmj"
Sep 17 20:52:01.232590 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.232534    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2272fdbe-f841-4620-8d45-531c5382ce08-node-exporter-textfile\") pod \"node-exporter-ddkmj\" (UID: \"2272fdbe-f841-4620-8d45-531c5382ce08\") " pod="openshift-monitoring/node-exporter-ddkmj"
Sep 17 20:52:01.233945 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.233927    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2272fdbe-f841-4620-8d45-531c5382ce08-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ddkmj\" (UID: \"2272fdbe-f841-4620-8d45-531c5382ce08\") " pod="openshift-monitoring/node-exporter-ddkmj"
Sep 17 20:52:01.234031 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.234010    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c393e135-8471-4c74-bdd2-6f4fe2da9e26-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-c2mkj\" (UID: \"c393e135-8471-4c74-bdd2-6f4fe2da9e26\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-c2mkj"
Sep 17 20:52:01.234167 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.234150    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c393e135-8471-4c74-bdd2-6f4fe2da9e26-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-c2mkj\" (UID: \"c393e135-8471-4c74-bdd2-6f4fe2da9e26\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-c2mkj"
Sep 17 20:52:01.239983 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.239956    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:01.241337 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.241312    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpxvk\" (UniqueName: \"kubernetes.io/projected/2272fdbe-f841-4620-8d45-531c5382ce08-kube-api-access-vpxvk\") pod \"node-exporter-ddkmj\" (UID: \"2272fdbe-f841-4620-8d45-531c5382ce08\") " pod="openshift-monitoring/node-exporter-ddkmj"
Sep 17 20:52:01.242197 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.242181    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fp27z\" (UniqueName: \"kubernetes.io/projected/c393e135-8471-4c74-bdd2-6f4fe2da9e26-kube-api-access-fp27z\") pod \"openshift-state-metrics-86c5bc79d6-c2mkj\" (UID: \"c393e135-8471-4c74-bdd2-6f4fe2da9e26\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-c2mkj"
Sep 17 20:52:01.319849 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.319819    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-c2mkj"
Sep 17 20:52:01.438348 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.438307    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:01.508401 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.508369    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:01.597578 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.597547    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-c2mkj"]
Sep 17 20:52:01.600465 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:52:01.600438    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc393e135_8471_4c74_bdd2_6f4fe2da9e26.slice/crio-32bf64148f5210448e739e9295a598ffb8f7b25d8fd4b19365a539ada87a3a40 WatchSource:0}: Error finding container 32bf64148f5210448e739e9295a598ffb8f7b25d8fd4b19365a539ada87a3a40: Status 404 returned error can't find the container with id 32bf64148f5210448e739e9295a598ffb8f7b25d8fd4b19365a539ada87a3a40
Sep 17 20:52:01.614134 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.614106    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:01.734967 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.734938    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2272fdbe-f841-4620-8d45-531c5382ce08-node-exporter-tls\") pod \"node-exporter-ddkmj\" (UID: \"2272fdbe-f841-4620-8d45-531c5382ce08\") " pod="openshift-monitoring/node-exporter-ddkmj"
Sep 17 20:52:01.737362 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.737337    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2272fdbe-f841-4620-8d45-531c5382ce08-node-exporter-tls\") pod \"node-exporter-ddkmj\" (UID: \"2272fdbe-f841-4620-8d45-531c5382ce08\") " pod="openshift-monitoring/node-exporter-ddkmj"
Sep 17 20:52:01.787153 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.787103    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-c2mkj" event={"ID":"c393e135-8471-4c74-bdd2-6f4fe2da9e26","Type":"ContainerStarted","Data":"1a7c360707d1bf0c1cbc163ea4e520b2d39c5f094dc9db593165f378b7ca7949"}
Sep 17 20:52:01.787315 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.787254    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-c2mkj" event={"ID":"c393e135-8471-4c74-bdd2-6f4fe2da9e26","Type":"ContainerStarted","Data":"2468cc8d9603e47c2a7953e873fde49fd8bd215efdf0d05c7db12010bc97b99e"}
Sep 17 20:52:01.787315 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:01.787280    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-c2mkj" event={"ID":"c393e135-8471-4c74-bdd2-6f4fe2da9e26","Type":"ContainerStarted","Data":"32bf64148f5210448e739e9295a598ffb8f7b25d8fd4b19365a539ada87a3a40"}
Sep 17 20:52:02.000577 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:02.000543    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ddkmj"
Sep 17 20:52:02.011653 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:52:02.011625    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2272fdbe_f841_4620_8d45_531c5382ce08.slice/crio-5add4577e53668f9a54890f3d6d4250810cb4c58c2597a391a108115cf7b571b WatchSource:0}: Error finding container 5add4577e53668f9a54890f3d6d4250810cb4c58c2597a391a108115cf7b571b: Status 404 returned error can't find the container with id 5add4577e53668f9a54890f3d6d4250810cb4c58c2597a391a108115cf7b571b
Sep 17 20:52:02.024182 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:02.024149    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:02.131408 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:02.131371    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:02.228874 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:02.228737    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:02.792067 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:02.792025    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ddkmj" event={"ID":"2272fdbe-f841-4620-8d45-531c5382ce08","Type":"ContainerStarted","Data":"5add4577e53668f9a54890f3d6d4250810cb4c58c2597a391a108115cf7b571b"}
Sep 17 20:52:03.073737 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:03.073708    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:03.186232 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:03.186191    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:03.388607 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:03.388526    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b4f78d9c4-crkjg"
Sep 17 20:52:03.388607 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:03.388583    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b4f78d9c4-crkjg"
Sep 17 20:52:03.393587 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:03.393565    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7b4f78d9c4-crkjg"
Sep 17 20:52:03.797614 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:03.797582    2567 generic.go:356] "Generic (PLEG): container finished" podID="2272fdbe-f841-4620-8d45-531c5382ce08" containerID="e6d8eb0bfa7537331b436a43d2574558c714eea535dc1211dcff8a62c05850a8" exitCode=0
Sep 17 20:52:03.797826 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:03.797687    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ddkmj" event={"ID":"2272fdbe-f841-4620-8d45-531c5382ce08","Type":"ContainerDied","Data":"e6d8eb0bfa7537331b436a43d2574558c714eea535dc1211dcff8a62c05850a8"}
Sep 17 20:52:03.800107 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:03.800085    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-c2mkj" event={"ID":"c393e135-8471-4c74-bdd2-6f4fe2da9e26","Type":"ContainerStarted","Data":"dec0729f21dac08b2c372b348e07408bff3841625cd8e0db0012673309f8bff2"}
Sep 17 20:52:03.804242 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:03.804222    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b4f78d9c4-crkjg"
Sep 17 20:52:03.841349 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:03.841306    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-c2mkj" podStartSLOduration=2.646214597 podStartE2EDuration="3.84128972s" totalImagesPullingTime="1.195075123s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:52:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:52:03.839841703 +0000 UTC m=+188.185443521" watchObservedRunningTime="2026-09-17 20:52:03.84128972 +0000 UTC m=+188.186891545"
Sep 17 20:52:04.023614 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.023578    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:04.117655 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.117557    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:04.121610 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.121582    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"]
Sep 17 20:52:04.125220 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.125203    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"
Sep 17 20:52:04.129375 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.129353    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 17 20:52:04.129487 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.129368    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 17 20:52:04.129487 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.129378    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 17 20:52:04.129608 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.129592    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 17 20:52:04.129880 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.129861    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-7k7v7\""
Sep 17 20:52:04.129952 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.129894    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 17 20:52:04.129952 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.129898    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-6pb203rt8dpk9\""
Sep 17 20:52:04.140372 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.140339    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"]
Sep 17 20:52:04.255974 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.255933    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6f1147bc-3278-4920-882d-8a05f33bc7c7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5dcb9bfd56-zxcf5\" (UID: \"6f1147bc-3278-4920-882d-8a05f33bc7c7\") " pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"
Sep 17 20:52:04.256119 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.255997    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6f1147bc-3278-4920-882d-8a05f33bc7c7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5dcb9bfd56-zxcf5\" (UID: \"6f1147bc-3278-4920-882d-8a05f33bc7c7\") " pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"
Sep 17 20:52:04.256119 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.256063    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6f1147bc-3278-4920-882d-8a05f33bc7c7-secret-grpc-tls\") pod \"thanos-querier-5dcb9bfd56-zxcf5\" (UID: \"6f1147bc-3278-4920-882d-8a05f33bc7c7\") " pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"
Sep 17 20:52:04.256199 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.256153    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm9vx\" (UniqueName: \"kubernetes.io/projected/6f1147bc-3278-4920-882d-8a05f33bc7c7-kube-api-access-fm9vx\") pod \"thanos-querier-5dcb9bfd56-zxcf5\" (UID: \"6f1147bc-3278-4920-882d-8a05f33bc7c7\") " pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"
Sep 17 20:52:04.256199 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.256180    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6f1147bc-3278-4920-882d-8a05f33bc7c7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5dcb9bfd56-zxcf5\" (UID: \"6f1147bc-3278-4920-882d-8a05f33bc7c7\") " pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"
Sep 17 20:52:04.256263 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.256244    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6f1147bc-3278-4920-882d-8a05f33bc7c7-secret-thanos-querier-tls\") pod \"thanos-querier-5dcb9bfd56-zxcf5\" (UID: \"6f1147bc-3278-4920-882d-8a05f33bc7c7\") " pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"
Sep 17 20:52:04.256295 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.256281    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6f1147bc-3278-4920-882d-8a05f33bc7c7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5dcb9bfd56-zxcf5\" (UID: \"6f1147bc-3278-4920-882d-8a05f33bc7c7\") " pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"
Sep 17 20:52:04.256325 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.256305    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f1147bc-3278-4920-882d-8a05f33bc7c7-metrics-client-ca\") pod \"thanos-querier-5dcb9bfd56-zxcf5\" (UID: \"6f1147bc-3278-4920-882d-8a05f33bc7c7\") " pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"
Sep 17 20:52:04.340566 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.340531    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:04.357616 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.357591    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm9vx\" (UniqueName: \"kubernetes.io/projected/6f1147bc-3278-4920-882d-8a05f33bc7c7-kube-api-access-fm9vx\") pod \"thanos-querier-5dcb9bfd56-zxcf5\" (UID: \"6f1147bc-3278-4920-882d-8a05f33bc7c7\") " pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"
Sep 17 20:52:04.357616 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.357618    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6f1147bc-3278-4920-882d-8a05f33bc7c7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5dcb9bfd56-zxcf5\" (UID: \"6f1147bc-3278-4920-882d-8a05f33bc7c7\") " pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"
Sep 17 20:52:04.357818 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.357645    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6f1147bc-3278-4920-882d-8a05f33bc7c7-secret-thanos-querier-tls\") pod \"thanos-querier-5dcb9bfd56-zxcf5\" (UID: \"6f1147bc-3278-4920-882d-8a05f33bc7c7\") " pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"
Sep 17 20:52:04.357818 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.357675    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6f1147bc-3278-4920-882d-8a05f33bc7c7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5dcb9bfd56-zxcf5\" (UID: \"6f1147bc-3278-4920-882d-8a05f33bc7c7\") " pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"
Sep 17 20:52:04.357818 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.357698    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f1147bc-3278-4920-882d-8a05f33bc7c7-metrics-client-ca\") pod \"thanos-querier-5dcb9bfd56-zxcf5\" (UID: \"6f1147bc-3278-4920-882d-8a05f33bc7c7\") " pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"
Sep 17 20:52:04.357818 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.357732    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6f1147bc-3278-4920-882d-8a05f33bc7c7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5dcb9bfd56-zxcf5\" (UID: \"6f1147bc-3278-4920-882d-8a05f33bc7c7\") " pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"
Sep 17 20:52:04.357818 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.357791    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6f1147bc-3278-4920-882d-8a05f33bc7c7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5dcb9bfd56-zxcf5\" (UID: \"6f1147bc-3278-4920-882d-8a05f33bc7c7\") " pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"
Sep 17 20:52:04.358069 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.357842    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6f1147bc-3278-4920-882d-8a05f33bc7c7-secret-grpc-tls\") pod \"thanos-querier-5dcb9bfd56-zxcf5\" (UID: \"6f1147bc-3278-4920-882d-8a05f33bc7c7\") " pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"
Sep 17 20:52:04.358544 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.358517    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f1147bc-3278-4920-882d-8a05f33bc7c7-metrics-client-ca\") pod \"thanos-querier-5dcb9bfd56-zxcf5\" (UID: \"6f1147bc-3278-4920-882d-8a05f33bc7c7\") " pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"
Sep 17 20:52:04.360567 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.360467    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6f1147bc-3278-4920-882d-8a05f33bc7c7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5dcb9bfd56-zxcf5\" (UID: \"6f1147bc-3278-4920-882d-8a05f33bc7c7\") " pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"
Sep 17 20:52:04.360567 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.360520    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6f1147bc-3278-4920-882d-8a05f33bc7c7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5dcb9bfd56-zxcf5\" (UID: \"6f1147bc-3278-4920-882d-8a05f33bc7c7\") " pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"
Sep 17 20:52:04.360994 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.360972    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6f1147bc-3278-4920-882d-8a05f33bc7c7-secret-thanos-querier-tls\") pod \"thanos-querier-5dcb9bfd56-zxcf5\" (UID: \"6f1147bc-3278-4920-882d-8a05f33bc7c7\") " pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"
Sep 17 20:52:04.361047 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.360986    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6f1147bc-3278-4920-882d-8a05f33bc7c7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5dcb9bfd56-zxcf5\" (UID: \"6f1147bc-3278-4920-882d-8a05f33bc7c7\") " pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"
Sep 17 20:52:04.361047 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.361002    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6f1147bc-3278-4920-882d-8a05f33bc7c7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5dcb9bfd56-zxcf5\" (UID: \"6f1147bc-3278-4920-882d-8a05f33bc7c7\") " pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"
Sep 17 20:52:04.361115 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.361091    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6f1147bc-3278-4920-882d-8a05f33bc7c7-secret-grpc-tls\") pod \"thanos-querier-5dcb9bfd56-zxcf5\" (UID: \"6f1147bc-3278-4920-882d-8a05f33bc7c7\") " pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"
Sep 17 20:52:04.369152 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.369085    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm9vx\" (UniqueName: \"kubernetes.io/projected/6f1147bc-3278-4920-882d-8a05f33bc7c7-kube-api-access-fm9vx\") pod \"thanos-querier-5dcb9bfd56-zxcf5\" (UID: \"6f1147bc-3278-4920-882d-8a05f33bc7c7\") " pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"
Sep 17 20:52:04.430282 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.430250    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:04.434254 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.434225    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"
Sep 17 20:52:04.722857 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.722832    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"]
Sep 17 20:52:04.724683 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:52:04.724642    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f1147bc_3278_4920_882d_8a05f33bc7c7.slice/crio-63926d1893992c80e88649d477d285eecc5ca8ee3d0df1370f18f576fc8f466f WatchSource:0}: Error finding container 63926d1893992c80e88649d477d285eecc5ca8ee3d0df1370f18f576fc8f466f: Status 404 returned error can't find the container with id 63926d1893992c80e88649d477d285eecc5ca8ee3d0df1370f18f576fc8f466f
Sep 17 20:52:04.805200 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.805157    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5" event={"ID":"6f1147bc-3278-4920-882d-8a05f33bc7c7","Type":"ContainerStarted","Data":"63926d1893992c80e88649d477d285eecc5ca8ee3d0df1370f18f576fc8f466f"}
Sep 17 20:52:04.807284 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.807254    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ddkmj" event={"ID":"2272fdbe-f841-4620-8d45-531c5382ce08","Type":"ContainerStarted","Data":"0bdc76f159075a46fc369e78d27d0444b4c6eb9c78b0793c48505d403dcf52ea"}
Sep 17 20:52:04.807409 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.807302    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ddkmj" event={"ID":"2272fdbe-f841-4620-8d45-531c5382ce08","Type":"ContainerStarted","Data":"79aeed722d4f332d88aec0154c5f7daee6a11fefac618acaf17d1a3a7c7768df"}
Sep 17 20:52:04.830075 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:04.830030    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ddkmj" podStartSLOduration=2.868272803 podStartE2EDuration="3.830011723s" totalImagesPullingTime="896.93613ms" totalInitContainerRuntime="64.80279ms" isStatefulPod=true podCreationTimestamp="2026-09-17 20:52:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:52:04.829690235 +0000 UTC m=+189.175292061" watchObservedRunningTime="2026-09-17 20:52:04.830011723 +0000 UTC m=+189.175613548"
Sep 17 20:52:05.781725 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:05.781685    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:05.818660 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:05.818617    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-prgc8"]
Sep 17 20:52:05.821866 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:05.821844    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-prgc8"
Sep 17 20:52:05.825278 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:05.825261    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-xfrrs\""
Sep 17 20:52:05.825787 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:05.825744    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 17 20:52:05.851815 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:05.851785    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-prgc8"]
Sep 17 20:52:05.870679 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:05.870645    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xh8r\" (UniqueName: \"kubernetes.io/projected/fbf8b156-01d4-45e7-bc62-a37b96168dd0-kube-api-access-6xh8r\") pod \"monitoring-plugin-5984459ff8-prgc8\" (UID: \"fbf8b156-01d4-45e7-bc62-a37b96168dd0\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-prgc8"
Sep 17 20:52:05.870847 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:05.870752    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fbf8b156-01d4-45e7-bc62-a37b96168dd0-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-prgc8\" (UID: \"fbf8b156-01d4-45e7-bc62-a37b96168dd0\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-prgc8"
Sep 17 20:52:05.925735 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:05.925684    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:05.972169 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:05.972132    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xh8r\" (UniqueName: \"kubernetes.io/projected/fbf8b156-01d4-45e7-bc62-a37b96168dd0-kube-api-access-6xh8r\") pod \"monitoring-plugin-5984459ff8-prgc8\" (UID: \"fbf8b156-01d4-45e7-bc62-a37b96168dd0\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-prgc8"
Sep 17 20:52:05.972327 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:05.972198    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fbf8b156-01d4-45e7-bc62-a37b96168dd0-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-prgc8\" (UID: \"fbf8b156-01d4-45e7-bc62-a37b96168dd0\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-prgc8"
Sep 17 20:52:05.972327 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:52:05.972299    2567 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 17 20:52:05.972453 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:52:05.972352    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbf8b156-01d4-45e7-bc62-a37b96168dd0-monitoring-plugin-cert podName:fbf8b156-01d4-45e7-bc62-a37b96168dd0 nodeName:}" failed. No retries permitted until 2026-09-17 20:52:06.472339107 +0000 UTC m=+190.817940910 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/fbf8b156-01d4-45e7-bc62-a37b96168dd0-monitoring-plugin-cert") pod "monitoring-plugin-5984459ff8-prgc8" (UID: "fbf8b156-01d4-45e7-bc62-a37b96168dd0") : secret "monitoring-plugin-cert" not found
Sep 17 20:52:06.001105 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:06.001075    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xh8r\" (UniqueName: \"kubernetes.io/projected/fbf8b156-01d4-45e7-bc62-a37b96168dd0-kube-api-access-6xh8r\") pod \"monitoring-plugin-5984459ff8-prgc8\" (UID: \"fbf8b156-01d4-45e7-bc62-a37b96168dd0\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-prgc8"
Sep 17 20:52:06.099451 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:06.099361    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:06.194293 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:06.194256    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:06.477124 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:06.477087    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fbf8b156-01d4-45e7-bc62-a37b96168dd0-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-prgc8\" (UID: \"fbf8b156-01d4-45e7-bc62-a37b96168dd0\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-prgc8"
Sep 17 20:52:06.479737 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:06.479711    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fbf8b156-01d4-45e7-bc62-a37b96168dd0-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-prgc8\" (UID: \"fbf8b156-01d4-45e7-bc62-a37b96168dd0\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-prgc8"
Sep 17 20:52:06.733937 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:06.733914    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-prgc8"
Sep 17 20:52:06.819488 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:06.819310    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5" event={"ID":"6f1147bc-3278-4920-882d-8a05f33bc7c7","Type":"ContainerStarted","Data":"c9dedbd89d8bd66ad95a2403da6f8e6e5af3bd34bd45cef4265995050a2a1b0f"}
Sep 17 20:52:06.819488 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:06.819399    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5" event={"ID":"6f1147bc-3278-4920-882d-8a05f33bc7c7","Type":"ContainerStarted","Data":"52313bd380e47d102df9b3ad17128ac157fb4a9d6348f0e96775ae7947502040"}
Sep 17 20:52:06.819488 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:06.819442    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5" event={"ID":"6f1147bc-3278-4920-882d-8a05f33bc7c7","Type":"ContainerStarted","Data":"892e42ef6c68269b6c021f1c74d675d5c6f93be690695bcb2df237ede7462ad7"}
Sep 17 20:52:07.007725 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.007650    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-prgc8"]
Sep 17 20:52:07.010895 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:52:07.010869    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbf8b156_01d4_45e7_bc62_a37b96168dd0.slice/crio-0620dc7a1dd67fdc7b53d881d3ac16863d7cc2df79f7c672a099051a81a99655 WatchSource:0}: Error finding container 0620dc7a1dd67fdc7b53d881d3ac16863d7cc2df79f7c672a099051a81a99655: Status 404 returned error can't find the container with id 0620dc7a1dd67fdc7b53d881d3ac16863d7cc2df79f7c672a099051a81a99655
Sep 17 20:52:07.358180 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.358089    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 17 20:52:07.362262 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.362238    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.366594 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.366014    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-hw2fb\""
Sep 17 20:52:07.366594 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.366073    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 17 20:52:07.366594 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.366100    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-e55qh19ij7uk\""
Sep 17 20:52:07.366594 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.366019    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 17 20:52:07.366594 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.366354    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 17 20:52:07.366594 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.366585    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 17 20:52:07.368182 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.368162    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 17 20:52:07.368271 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.368180    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 17 20:52:07.368493 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.368479    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 17 20:52:07.368614 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.368600    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 17 20:52:07.369077 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.369054    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 17 20:52:07.369635 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.369616    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 17 20:52:07.370738 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.370704    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 17 20:52:07.370738 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.370730    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 17 20:52:07.370903 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.370788    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 17 20:52:07.370991 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.370972    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 17 20:52:07.375638 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.375616    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 17 20:52:07.381980 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.380014    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:07.383364 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.383303    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 17 20:52:07.486153 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.486116    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.486340 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.486167    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/77092fca-4fd9-4578-851b-f0bb1cdff864-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.486340 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.486189    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.486340 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.486215    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.486340 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.486239    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/77092fca-4fd9-4578-851b-f0bb1cdff864-config-out\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.486340 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.486267    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.486340 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.486289    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.486340 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.486313    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq8hd\" (UniqueName: \"kubernetes.io/projected/77092fca-4fd9-4578-851b-f0bb1cdff864-kube-api-access-mq8hd\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.486621 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.486379    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.486621 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.486452    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.486621 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.486480    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.486621 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.486524    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.486621 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.486543    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.486621 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.486565    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.486621 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.486597    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-config\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.486967 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.486671    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/77092fca-4fd9-4578-851b-f0bb1cdff864-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.486967 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.486717    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.486967 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.486774    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-web-config\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.486967 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.486820    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.486967 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.486844    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.588241 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.588203    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-web-config\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.588329 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.588260    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.588329 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.588292    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.588329 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.588323    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.588461 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.588355    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/77092fca-4fd9-4578-851b-f0bb1cdff864-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.588461 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.588380    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.588461 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.588404    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.588461 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.588434    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/77092fca-4fd9-4578-851b-f0bb1cdff864-config-out\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.588667 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.588463    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.588667 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.588488    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.588667 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.588510    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq8hd\" (UniqueName: \"kubernetes.io/projected/77092fca-4fd9-4578-851b-f0bb1cdff864-kube-api-access-mq8hd\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.588667 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.588549    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.588667 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.588599    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.588667 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.588633    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.588667 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.588667    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.589040 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.588694    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.589040 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.588729    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.589040 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.588788    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-config\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.589040 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.588816    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/77092fca-4fd9-4578-851b-f0bb1cdff864-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.589040 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.588841    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.589306 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.589179    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.589570 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.589545    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.590258 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.590096    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.590478 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.590455    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.591329 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.591199    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-web-config\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.591431 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.591346    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.592433 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.592403    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.593190 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.592602    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/77092fca-4fd9-4578-851b-f0bb1cdff864-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.593591 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.593545    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.594632 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.594599    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.594904 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.594885    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/77092fca-4fd9-4578-851b-f0bb1cdff864-config-out\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.596065 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.596042    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.597427 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.597402    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.598533 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.598484    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/77092fca-4fd9-4578-851b-f0bb1cdff864-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.599117 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.599093    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.599678 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.599385    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.599782 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.599677    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq8hd\" (UniqueName: \"kubernetes.io/projected/77092fca-4fd9-4578-851b-f0bb1cdff864-kube-api-access-mq8hd\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.600603 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.600575    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-config\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.601866 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.601118    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.601866 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.601793    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.676920 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.676896    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:07.830508 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.830404    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5" event={"ID":"6f1147bc-3278-4920-882d-8a05f33bc7c7","Type":"ContainerStarted","Data":"d42afa80b4ae38708e88759342edd8f031b4e584d6af948fa1c24b4bc9b98404"}
Sep 17 20:52:07.830508 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.830495    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5" event={"ID":"6f1147bc-3278-4920-882d-8a05f33bc7c7","Type":"ContainerStarted","Data":"3e3abf14b085fe204961e80b3c28f637438277c50a77ecfb8ceb41b2cab07d68"}
Sep 17 20:52:07.831055 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.830542    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5" event={"ID":"6f1147bc-3278-4920-882d-8a05f33bc7c7","Type":"ContainerStarted","Data":"9441b88ecfa5c92177d75a1ff741232cc301068d5b69907f9c5505ef1c4f2054"}
Sep 17 20:52:07.831055 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.830686    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"
Sep 17 20:52:07.832018 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.831990    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-prgc8" event={"ID":"fbf8b156-01d4-45e7-bc62-a37b96168dd0","Type":"ContainerStarted","Data":"0620dc7a1dd67fdc7b53d881d3ac16863d7cc2df79f7c672a099051a81a99655"}
Sep 17 20:52:07.855883 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.855825    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5" podStartSLOduration=1.254875844 podStartE2EDuration="3.855802819s" totalImagesPullingTime="2.600926975s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:52:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:52:07.85431659 +0000 UTC m=+192.199918416" watchObservedRunningTime="2026-09-17 20:52:07.855802819 +0000 UTC m=+192.201404643"
Sep 17 20:52:07.991655 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:07.991616    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 17 20:52:07.996545 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:52:07.996518    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77092fca_4fd9_4578_851b_f0bb1cdff864.slice/crio-15c52658ad530b7772972ef51300b37febdcadca219e89761ec07574319cbb9a WatchSource:0}: Error finding container 15c52658ad530b7772972ef51300b37febdcadca219e89761ec07574319cbb9a: Status 404 returned error can't find the container with id 15c52658ad530b7772972ef51300b37febdcadca219e89761ec07574319cbb9a
Sep 17 20:52:08.551450 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:08.551420    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:08.639025 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:08.638965    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:08.750523 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:08.750451    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-854db97696-6sllh"]
Sep 17 20:52:08.754982 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:08.754961    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:52:08.838502 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:08.838455    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-prgc8" event={"ID":"fbf8b156-01d4-45e7-bc62-a37b96168dd0","Type":"ContainerStarted","Data":"ad2c0796605b03b3734a9f075330865ea66e9e35d7f319721c822d7af89fb60e"}
Sep 17 20:52:08.838965 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:08.838526    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-prgc8"
Sep 17 20:52:08.839950 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:08.839879    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"77092fca-4fd9-4578-851b-f0bb1cdff864","Type":"ContainerStarted","Data":"15c52658ad530b7772972ef51300b37febdcadca219e89761ec07574319cbb9a"}
Sep 17 20:52:08.843107 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:08.843080    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:08.845209 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:08.845188    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-prgc8"
Sep 17 20:52:08.856072 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:08.856023    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5984459ff8-prgc8" podStartSLOduration=2.404045082 podStartE2EDuration="3.856006272s" totalImagesPullingTime="1.45196119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:52:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:52:08.854901952 +0000 UTC m=+193.200503781" watchObservedRunningTime="2026-09-17 20:52:08.856006272 +0000 UTC m=+193.201608096"
Sep 17 20:52:08.944454 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:08.944421    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:09.660592 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:09.660551    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:09.725471 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:09.725440    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:09.845442 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:09.845413    2567 generic.go:356] "Generic (PLEG): container finished" podID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerID="6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98" exitCode=0
Sep 17 20:52:09.845816 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:09.845495    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"77092fca-4fd9-4578-851b-f0bb1cdff864","Type":"ContainerDied","Data":"6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98"}
Sep 17 20:52:13.845697 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:13.845667    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5dcb9bfd56-zxcf5"
Sep 17 20:52:13.866490 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:13.866456    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"77092fca-4fd9-4578-851b-f0bb1cdff864","Type":"ContainerStarted","Data":"d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d"}
Sep 17 20:52:13.866646 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:13.866519    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"77092fca-4fd9-4578-851b-f0bb1cdff864","Type":"ContainerStarted","Data":"f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1"}
Sep 17 20:52:13.866646 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:13.866553    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"77092fca-4fd9-4578-851b-f0bb1cdff864","Type":"ContainerStarted","Data":"65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0"}
Sep 17 20:52:13.866646 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:13.866579    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"77092fca-4fd9-4578-851b-f0bb1cdff864","Type":"ContainerStarted","Data":"37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec"}
Sep 17 20:52:13.866646 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:13.866606    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"77092fca-4fd9-4578-851b-f0bb1cdff864","Type":"ContainerStarted","Data":"3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702"}
Sep 17 20:52:13.866646 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:13.866632    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"77092fca-4fd9-4578-851b-f0bb1cdff864","Type":"ContainerStarted","Data":"a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3"}
Sep 17 20:52:13.902382 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:13.902328    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.735651923 podStartE2EDuration="6.902313173s" totalImagesPullingTime="4.147855005s" totalInitContainerRuntime="18.806245ms" isStatefulPod=false podCreationTimestamp="2026-09-17 20:52:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:52:13.900637953 +0000 UTC m=+198.246239779" watchObservedRunningTime="2026-09-17 20:52:13.902313173 +0000 UTC m=+198.247914998"
Sep 17 20:52:17.677792 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:17.677744    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:52:18.159964 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:18.159933    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b4f78d9c4-crkjg"]
Sep 17 20:52:18.283051 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:18.283013    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:18.385335 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:18.385286    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:18.484776 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:18.484727    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:19.172330 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:19.172291    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:19.273608 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:19.273573    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:26.217000 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:26.216955    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:26.291465 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:26.291429    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:28.760509 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:28.760477    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:28.862592 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:28.862558    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:33.769420 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:33.769378    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-854db97696-6sllh" podUID="662880e7-4dd4-4556-a2db-ba6b415784aa" containerName="registry" containerID="cri-o://8b263aa8e82f74cc129816aecdec7a49f5333b568be8a5085ce417062333badb" gracePeriod=30
Sep 17 20:52:33.949451 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:33.949416    2567 generic.go:356] "Generic (PLEG): container finished" podID="662880e7-4dd4-4556-a2db-ba6b415784aa" containerID="8b263aa8e82f74cc129816aecdec7a49f5333b568be8a5085ce417062333badb" exitCode=0
Sep 17 20:52:33.949589 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:33.949497    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-854db97696-6sllh" event={"ID":"662880e7-4dd4-4556-a2db-ba6b415784aa","Type":"ContainerDied","Data":"8b263aa8e82f74cc129816aecdec7a49f5333b568be8a5085ce417062333badb"}
Sep 17 20:52:34.006132 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:34.006105    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:52:34.024809 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:34.024720    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-bound-sa-token\") pod \"662880e7-4dd4-4556-a2db-ba6b415784aa\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") "
Sep 17 20:52:34.024809 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:34.024752    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bg2bm\" (UniqueName: \"kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-kube-api-access-bg2bm\") pod \"662880e7-4dd4-4556-a2db-ba6b415784aa\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") "
Sep 17 20:52:34.024809 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:34.024781    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/662880e7-4dd4-4556-a2db-ba6b415784aa-trusted-ca\") pod \"662880e7-4dd4-4556-a2db-ba6b415784aa\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") "
Sep 17 20:52:34.024809 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:34.024812    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/662880e7-4dd4-4556-a2db-ba6b415784aa-ca-trust-extracted\") pod \"662880e7-4dd4-4556-a2db-ba6b415784aa\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") "
Sep 17 20:52:34.025113 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:34.024845    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-certificates\") pod \"662880e7-4dd4-4556-a2db-ba6b415784aa\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") "
Sep 17 20:52:34.025113 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:34.024890    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/662880e7-4dd4-4556-a2db-ba6b415784aa-installation-pull-secrets\") pod \"662880e7-4dd4-4556-a2db-ba6b415784aa\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") "
Sep 17 20:52:34.025113 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:34.024924    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-tls\") pod \"662880e7-4dd4-4556-a2db-ba6b415784aa\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") "
Sep 17 20:52:34.025113 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:34.024960    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/662880e7-4dd4-4556-a2db-ba6b415784aa-image-registry-private-configuration\") pod \"662880e7-4dd4-4556-a2db-ba6b415784aa\" (UID: \"662880e7-4dd4-4556-a2db-ba6b415784aa\") "
Sep 17 20:52:34.025317 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:34.025267    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/662880e7-4dd4-4556-a2db-ba6b415784aa-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "662880e7-4dd4-4556-a2db-ba6b415784aa" (UID: "662880e7-4dd4-4556-a2db-ba6b415784aa"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 20:52:34.025709 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:34.025658    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "662880e7-4dd4-4556-a2db-ba6b415784aa" (UID: "662880e7-4dd4-4556-a2db-ba6b415784aa"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 20:52:34.027327 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:34.027281    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "662880e7-4dd4-4556-a2db-ba6b415784aa" (UID: "662880e7-4dd4-4556-a2db-ba6b415784aa"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 20:52:34.027451 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:34.027347    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-kube-api-access-bg2bm" (OuterVolumeSpecName: "kube-api-access-bg2bm") pod "662880e7-4dd4-4556-a2db-ba6b415784aa" (UID: "662880e7-4dd4-4556-a2db-ba6b415784aa"). InnerVolumeSpecName "kube-api-access-bg2bm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 20:52:34.027526 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:34.027495    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/662880e7-4dd4-4556-a2db-ba6b415784aa-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "662880e7-4dd4-4556-a2db-ba6b415784aa" (UID: "662880e7-4dd4-4556-a2db-ba6b415784aa"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 20:52:34.027576 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:34.027554    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/662880e7-4dd4-4556-a2db-ba6b415784aa-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "662880e7-4dd4-4556-a2db-ba6b415784aa" (UID: "662880e7-4dd4-4556-a2db-ba6b415784aa"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 20:52:34.027711 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:34.027688    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "662880e7-4dd4-4556-a2db-ba6b415784aa" (UID: "662880e7-4dd4-4556-a2db-ba6b415784aa"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 20:52:34.034668 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:34.034641    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/662880e7-4dd4-4556-a2db-ba6b415784aa-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "662880e7-4dd4-4556-a2db-ba6b415784aa" (UID: "662880e7-4dd4-4556-a2db-ba6b415784aa"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 20:52:34.126440 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:34.126405    2567 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/662880e7-4dd4-4556-a2db-ba6b415784aa-installation-pull-secrets\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:52:34.126440 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:34.126434    2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-tls\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:52:34.126440 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:34.126445    2567 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/662880e7-4dd4-4556-a2db-ba6b415784aa-image-registry-private-configuration\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:52:34.126668 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:34.126454    2567 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-bound-sa-token\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:52:34.126668 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:34.126462    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bg2bm\" (UniqueName: \"kubernetes.io/projected/662880e7-4dd4-4556-a2db-ba6b415784aa-kube-api-access-bg2bm\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:52:34.126668 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:34.126470    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/662880e7-4dd4-4556-a2db-ba6b415784aa-trusted-ca\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:52:34.126668 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:34.126478    2567 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/662880e7-4dd4-4556-a2db-ba6b415784aa-ca-trust-extracted\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:52:34.126668 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:34.126487    2567 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/662880e7-4dd4-4556-a2db-ba6b415784aa-registry-certificates\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:52:34.955128 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:34.955092    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-854db97696-6sllh"
Sep 17 20:52:34.955128 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:34.955116    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-854db97696-6sllh" event={"ID":"662880e7-4dd4-4556-a2db-ba6b415784aa","Type":"ContainerDied","Data":"3c2b4e91b03b5e7af0654ceac6edc8f2ba3f70eb845adb02637d2f0f1fcef058"}
Sep 17 20:52:34.955637 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:34.955182    2567 scope.go:117] "RemoveContainer" containerID="8b263aa8e82f74cc129816aecdec7a49f5333b568be8a5085ce417062333badb"
Sep 17 20:52:34.977693 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:34.977662    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-854db97696-6sllh"]
Sep 17 20:52:34.983260 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:34.983236    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-854db97696-6sllh"]
Sep 17 20:52:35.276945 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:35.276903    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:35.417559 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:35.417524    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:36.169225 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:36.169195    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="662880e7-4dd4-4556-a2db-ba6b415784aa" path="/var/lib/kubelet/pods/662880e7-4dd4-4556-a2db-ba6b415784aa/volumes"
Sep 17 20:52:43.179129 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:43.179084    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7b4f78d9c4-crkjg" podUID="8e8fb726-7627-4b4a-820f-af3caa14b16d" containerName="console" containerID="cri-o://84faac994db76cd91dfd5d7fcf38550d8f007c0d9648acbd6be96af7af9c5d69" gracePeriod=15
Sep 17 20:52:43.421295 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:43.421268    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b4f78d9c4-crkjg"
Sep 17 20:52:43.501328 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:43.501289    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8e8fb726-7627-4b4a-820f-af3caa14b16d-oauth-serving-cert\") pod \"8e8fb726-7627-4b4a-820f-af3caa14b16d\" (UID: \"8e8fb726-7627-4b4a-820f-af3caa14b16d\") "
Sep 17 20:52:43.501328 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:43.501335    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tl8tv\" (UniqueName: \"kubernetes.io/projected/8e8fb726-7627-4b4a-820f-af3caa14b16d-kube-api-access-tl8tv\") pod \"8e8fb726-7627-4b4a-820f-af3caa14b16d\" (UID: \"8e8fb726-7627-4b4a-820f-af3caa14b16d\") "
Sep 17 20:52:43.501542 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:43.501356    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8e8fb726-7627-4b4a-820f-af3caa14b16d-service-ca\") pod \"8e8fb726-7627-4b4a-820f-af3caa14b16d\" (UID: \"8e8fb726-7627-4b4a-820f-af3caa14b16d\") "
Sep 17 20:52:43.501542 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:43.501377    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8e8fb726-7627-4b4a-820f-af3caa14b16d-console-oauth-config\") pod \"8e8fb726-7627-4b4a-820f-af3caa14b16d\" (UID: \"8e8fb726-7627-4b4a-820f-af3caa14b16d\") "
Sep 17 20:52:43.501542 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:43.501394    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8e8fb726-7627-4b4a-820f-af3caa14b16d-console-config\") pod \"8e8fb726-7627-4b4a-820f-af3caa14b16d\" (UID: \"8e8fb726-7627-4b4a-820f-af3caa14b16d\") "
Sep 17 20:52:43.501542 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:43.501448    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8e8fb726-7627-4b4a-820f-af3caa14b16d-console-serving-cert\") pod \"8e8fb726-7627-4b4a-820f-af3caa14b16d\" (UID: \"8e8fb726-7627-4b4a-820f-af3caa14b16d\") "
Sep 17 20:52:43.501816 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:43.501789    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e8fb726-7627-4b4a-820f-af3caa14b16d-service-ca" (OuterVolumeSpecName: "service-ca") pod "8e8fb726-7627-4b4a-820f-af3caa14b16d" (UID: "8e8fb726-7627-4b4a-820f-af3caa14b16d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 20:52:43.501906 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:43.501800    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e8fb726-7627-4b4a-820f-af3caa14b16d-console-config" (OuterVolumeSpecName: "console-config") pod "8e8fb726-7627-4b4a-820f-af3caa14b16d" (UID: "8e8fb726-7627-4b4a-820f-af3caa14b16d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 20:52:43.502014 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:43.501986    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e8fb726-7627-4b4a-820f-af3caa14b16d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8e8fb726-7627-4b4a-820f-af3caa14b16d" (UID: "8e8fb726-7627-4b4a-820f-af3caa14b16d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 20:52:43.503593 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:43.503564    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e8fb726-7627-4b4a-820f-af3caa14b16d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8e8fb726-7627-4b4a-820f-af3caa14b16d" (UID: "8e8fb726-7627-4b4a-820f-af3caa14b16d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 20:52:43.503678 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:43.503597    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e8fb726-7627-4b4a-820f-af3caa14b16d-kube-api-access-tl8tv" (OuterVolumeSpecName: "kube-api-access-tl8tv") pod "8e8fb726-7627-4b4a-820f-af3caa14b16d" (UID: "8e8fb726-7627-4b4a-820f-af3caa14b16d"). InnerVolumeSpecName "kube-api-access-tl8tv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 20:52:43.503678 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:43.503607    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e8fb726-7627-4b4a-820f-af3caa14b16d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8e8fb726-7627-4b4a-820f-af3caa14b16d" (UID: "8e8fb726-7627-4b4a-820f-af3caa14b16d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 20:52:43.602651 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:43.602624    2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8e8fb726-7627-4b4a-820f-af3caa14b16d-oauth-serving-cert\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:52:43.602651 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:43.602647    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tl8tv\" (UniqueName: \"kubernetes.io/projected/8e8fb726-7627-4b4a-820f-af3caa14b16d-kube-api-access-tl8tv\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:52:43.602831 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:43.602657    2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8e8fb726-7627-4b4a-820f-af3caa14b16d-service-ca\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:52:43.602831 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:43.602667    2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8e8fb726-7627-4b4a-820f-af3caa14b16d-console-oauth-config\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:52:43.602831 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:43.602675    2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8e8fb726-7627-4b4a-820f-af3caa14b16d-console-config\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:52:43.602831 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:43.602683    2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8e8fb726-7627-4b4a-820f-af3caa14b16d-console-serving-cert\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:52:43.754162 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:43.754081    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:43.855088 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:43.855056    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:43.957525 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:43.957486    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:43.990822 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:43.990790    2567 generic.go:356] "Generic (PLEG): container finished" podID="8e8fb726-7627-4b4a-820f-af3caa14b16d" containerID="84faac994db76cd91dfd5d7fcf38550d8f007c0d9648acbd6be96af7af9c5d69" exitCode=0
Sep 17 20:52:43.990987 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:43.990845    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b4f78d9c4-crkjg"
Sep 17 20:52:43.990987 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:43.990866    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b4f78d9c4-crkjg" event={"ID":"8e8fb726-7627-4b4a-820f-af3caa14b16d","Type":"ContainerDied","Data":"84faac994db76cd91dfd5d7fcf38550d8f007c0d9648acbd6be96af7af9c5d69"}
Sep 17 20:52:43.990987 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:43.990926    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b4f78d9c4-crkjg" event={"ID":"8e8fb726-7627-4b4a-820f-af3caa14b16d","Type":"ContainerDied","Data":"ebd4ff569a6caee0ddf46e500e28e0782adf22b5923ba1ee613ccaeda2fcb3da"}
Sep 17 20:52:43.990987 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:43.990959    2567 scope.go:117] "RemoveContainer" containerID="84faac994db76cd91dfd5d7fcf38550d8f007c0d9648acbd6be96af7af9c5d69"
Sep 17 20:52:44.000286 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:44.000267    2567 scope.go:117] "RemoveContainer" containerID="84faac994db76cd91dfd5d7fcf38550d8f007c0d9648acbd6be96af7af9c5d69"
Sep 17 20:52:44.000561 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:52:44.000543    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84faac994db76cd91dfd5d7fcf38550d8f007c0d9648acbd6be96af7af9c5d69\": container with ID starting with 84faac994db76cd91dfd5d7fcf38550d8f007c0d9648acbd6be96af7af9c5d69 not found: ID does not exist" containerID="84faac994db76cd91dfd5d7fcf38550d8f007c0d9648acbd6be96af7af9c5d69"
Sep 17 20:52:44.000624 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:44.000568    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84faac994db76cd91dfd5d7fcf38550d8f007c0d9648acbd6be96af7af9c5d69"} err="failed to get container status \"84faac994db76cd91dfd5d7fcf38550d8f007c0d9648acbd6be96af7af9c5d69\": rpc error: code = NotFound desc = could not find container \"84faac994db76cd91dfd5d7fcf38550d8f007c0d9648acbd6be96af7af9c5d69\": container with ID starting with 84faac994db76cd91dfd5d7fcf38550d8f007c0d9648acbd6be96af7af9c5d69 not found: ID does not exist"
Sep 17 20:52:44.012083 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:44.012031    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b4f78d9c4-crkjg"]
Sep 17 20:52:44.017460 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:44.017441    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b4f78d9c4-crkjg"]
Sep 17 20:52:44.169114 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:52:44.169079    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e8fb726-7627-4b4a-820f-af3caa14b16d" path="/var/lib/kubelet/pods/8e8fb726-7627-4b4a-820f-af3caa14b16d/volumes"
Sep 17 20:53:01.180391 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:01.180356    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:53:07.677902 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:07.677865    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:07.696957 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:07.696934    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:07.997916 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:07.997884    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-metrics-certs\") pod \"network-metrics-daemon-jhhl5\" (UID: \"b4f3e778-0cec-45fe-be1b-a8789d63c1d7\") " pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:53:08.000178 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:08.000156    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4f3e778-0cec-45fe-be1b-a8789d63c1d7-metrics-certs\") pod \"network-metrics-daemon-jhhl5\" (UID: \"b4f3e778-0cec-45fe-be1b-a8789d63c1d7\") " pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:53:08.068840 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:08.068815    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6gfd5\""
Sep 17 20:53:08.076113 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:08.076092    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhhl5"
Sep 17 20:53:08.095832 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:08.095805    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:08.346989 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:08.346965    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jhhl5"]
Sep 17 20:53:08.349580 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:53:08.349551    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4f3e778_0cec_45fe_be1b_a8789d63c1d7.slice/crio-349f0bda3070e50d45214cd92fd2e5005966e626f2cc89ac4fe361f7395faf7b WatchSource:0}: Error finding container 349f0bda3070e50d45214cd92fd2e5005966e626f2cc89ac4fe361f7395faf7b: Status 404 returned error can't find the container with id 349f0bda3070e50d45214cd92fd2e5005966e626f2cc89ac4fe361f7395faf7b
Sep 17 20:53:09.086636 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:09.086606    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jhhl5" event={"ID":"b4f3e778-0cec-45fe-be1b-a8789d63c1d7","Type":"ContainerStarted","Data":"349f0bda3070e50d45214cd92fd2e5005966e626f2cc89ac4fe361f7395faf7b"}
Sep 17 20:53:10.092425 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:10.092391    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jhhl5" event={"ID":"b4f3e778-0cec-45fe-be1b-a8789d63c1d7","Type":"ContainerStarted","Data":"b8e9a2da815eb5137643f61a21aff44caf1f6aabb0393fe4630e74c2f6ac3be3"}
Sep 17 20:53:10.092799 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:10.092482    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jhhl5" event={"ID":"b4f3e778-0cec-45fe-be1b-a8789d63c1d7","Type":"ContainerStarted","Data":"d214f282e27aa6c2c4c8d2d1648fc08c693dbcc64c2284b427ddfaa66e7d0c9b"}
Sep 17 20:53:10.111578 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:10.111528    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jhhl5" podStartSLOduration=253.187399191 podStartE2EDuration="4m14.11151254s" totalImagesPullingTime="924.113349ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:53:10.110373327 +0000 UTC m=+254.455975162" watchObservedRunningTime="2026-09-17 20:53:10.11151254 +0000 UTC m=+254.457114364"
Sep 17 20:53:10.215398 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:10.215360    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:53:10.281347 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:10.281305    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:53:20.366837 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:20.366732    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:53:20.713687 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:20.713644    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:53:25.777070 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:25.777029    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 17 20:53:25.777566 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:25.777389    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerName="prometheus" containerID="cri-o://a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3" gracePeriod=600
Sep 17 20:53:25.777566 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:25.777406    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerName="kube-rbac-proxy" containerID="cri-o://f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1" gracePeriod=600
Sep 17 20:53:25.777566 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:25.777446    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerName="config-reloader" containerID="cri-o://3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702" gracePeriod=600
Sep 17 20:53:25.777566 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:25.777464    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerName="thanos-sidecar" containerID="cri-o://37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec" gracePeriod=600
Sep 17 20:53:25.777566 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:25.777464    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerName="kube-rbac-proxy-web" containerID="cri-o://65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0" gracePeriod=600
Sep 17 20:53:25.777836 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:25.777458    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerName="kube-rbac-proxy-thanos" containerID="cri-o://d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d" gracePeriod=600
Sep 17 20:53:26.027059 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.027031    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.142158 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.142072    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-prometheus-k8s-rulefiles-2\") pod \"77092fca-4fd9-4578-851b-f0bb1cdff864\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") "
Sep 17 20:53:26.142158 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.142106    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-config\") pod \"77092fca-4fd9-4578-851b-f0bb1cdff864\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") "
Sep 17 20:53:26.142158 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.142134    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-configmap-kubelet-serving-ca-bundle\") pod \"77092fca-4fd9-4578-851b-f0bb1cdff864\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") "
Sep 17 20:53:26.142158 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.142157    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"77092fca-4fd9-4578-851b-f0bb1cdff864\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") "
Sep 17 20:53:26.142462 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.142174    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-thanos-prometheus-http-client-file\") pod \"77092fca-4fd9-4578-851b-f0bb1cdff864\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") "
Sep 17 20:53:26.142462 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.142201    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/77092fca-4fd9-4578-851b-f0bb1cdff864-config-out\") pod \"77092fca-4fd9-4578-851b-f0bb1cdff864\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") "
Sep 17 20:53:26.142462 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.142224    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-web-config\") pod \"77092fca-4fd9-4578-851b-f0bb1cdff864\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") "
Sep 17 20:53:26.142462 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.142249    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/77092fca-4fd9-4578-851b-f0bb1cdff864-tls-assets\") pod \"77092fca-4fd9-4578-851b-f0bb1cdff864\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") "
Sep 17 20:53:26.142462 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.142266    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/77092fca-4fd9-4578-851b-f0bb1cdff864-prometheus-k8s-db\") pod \"77092fca-4fd9-4578-851b-f0bb1cdff864\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") "
Sep 17 20:53:26.142462 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.142300    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"77092fca-4fd9-4578-851b-f0bb1cdff864\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") "
Sep 17 20:53:26.142462 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.142326    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-configmap-serving-certs-ca-bundle\") pod \"77092fca-4fd9-4578-851b-f0bb1cdff864\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") "
Sep 17 20:53:26.142462 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.142350    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-kube-rbac-proxy\") pod \"77092fca-4fd9-4578-851b-f0bb1cdff864\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") "
Sep 17 20:53:26.142462 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.142405    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-prometheus-k8s-rulefiles-0\") pod \"77092fca-4fd9-4578-851b-f0bb1cdff864\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") "
Sep 17 20:53:26.142462 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.142434    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mq8hd\" (UniqueName: \"kubernetes.io/projected/77092fca-4fd9-4578-851b-f0bb1cdff864-kube-api-access-mq8hd\") pod \"77092fca-4fd9-4578-851b-f0bb1cdff864\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") "
Sep 17 20:53:26.142462 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.142458    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-prometheus-k8s-tls\") pod \"77092fca-4fd9-4578-851b-f0bb1cdff864\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") "
Sep 17 20:53:26.143005 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.142486    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-metrics-client-certs\") pod \"77092fca-4fd9-4578-851b-f0bb1cdff864\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") "
Sep 17 20:53:26.143005 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.142500    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "77092fca-4fd9-4578-851b-f0bb1cdff864" (UID: "77092fca-4fd9-4578-851b-f0bb1cdff864"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 20:53:26.143005 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.142516    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-prometheus-k8s-rulefiles-1\") pod \"77092fca-4fd9-4578-851b-f0bb1cdff864\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") "
Sep 17 20:53:26.143005 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.142579    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-prometheus-trusted-ca-bundle\") pod \"77092fca-4fd9-4578-851b-f0bb1cdff864\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") "
Sep 17 20:53:26.143005 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.142606    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-grpc-tls\") pod \"77092fca-4fd9-4578-851b-f0bb1cdff864\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") "
Sep 17 20:53:26.143005 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.142631    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-configmap-metrics-client-ca\") pod \"77092fca-4fd9-4578-851b-f0bb1cdff864\" (UID: \"77092fca-4fd9-4578-851b-f0bb1cdff864\") "
Sep 17 20:53:26.143005 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.142865    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "77092fca-4fd9-4578-851b-f0bb1cdff864" (UID: "77092fca-4fd9-4578-851b-f0bb1cdff864"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 20:53:26.143005 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.142911    2567 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:53:26.143712 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.143653    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "77092fca-4fd9-4578-851b-f0bb1cdff864" (UID: "77092fca-4fd9-4578-851b-f0bb1cdff864"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 20:53:26.143845 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.143746    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77092fca-4fd9-4578-851b-f0bb1cdff864-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "77092fca-4fd9-4578-851b-f0bb1cdff864" (UID: "77092fca-4fd9-4578-851b-f0bb1cdff864"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 20:53:26.143845 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.143752    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "77092fca-4fd9-4578-851b-f0bb1cdff864" (UID: "77092fca-4fd9-4578-851b-f0bb1cdff864"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 20:53:26.144655 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.144628    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "77092fca-4fd9-4578-851b-f0bb1cdff864" (UID: "77092fca-4fd9-4578-851b-f0bb1cdff864"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 20:53:26.145198 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.145140    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "77092fca-4fd9-4578-851b-f0bb1cdff864" (UID: "77092fca-4fd9-4578-851b-f0bb1cdff864"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 20:53:26.145812 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.145623    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "77092fca-4fd9-4578-851b-f0bb1cdff864" (UID: "77092fca-4fd9-4578-851b-f0bb1cdff864"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 20:53:26.145812 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.145777    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-config" (OuterVolumeSpecName: "config") pod "77092fca-4fd9-4578-851b-f0bb1cdff864" (UID: "77092fca-4fd9-4578-851b-f0bb1cdff864"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 20:53:26.145985 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.145912    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "77092fca-4fd9-4578-851b-f0bb1cdff864" (UID: "77092fca-4fd9-4578-851b-f0bb1cdff864"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 20:53:26.146666 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.146621    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "77092fca-4fd9-4578-851b-f0bb1cdff864" (UID: "77092fca-4fd9-4578-851b-f0bb1cdff864"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 20:53:26.146941 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.146919    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77092fca-4fd9-4578-851b-f0bb1cdff864-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "77092fca-4fd9-4578-851b-f0bb1cdff864" (UID: "77092fca-4fd9-4578-851b-f0bb1cdff864"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 20:53:26.147042 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.147022    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "77092fca-4fd9-4578-851b-f0bb1cdff864" (UID: "77092fca-4fd9-4578-851b-f0bb1cdff864"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 20:53:26.147259 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.147234    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "77092fca-4fd9-4578-851b-f0bb1cdff864" (UID: "77092fca-4fd9-4578-851b-f0bb1cdff864"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 20:53:26.147344 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.147310    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "77092fca-4fd9-4578-851b-f0bb1cdff864" (UID: "77092fca-4fd9-4578-851b-f0bb1cdff864"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 20:53:26.147398 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.147383    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77092fca-4fd9-4578-851b-f0bb1cdff864-config-out" (OuterVolumeSpecName: "config-out") pod "77092fca-4fd9-4578-851b-f0bb1cdff864" (UID: "77092fca-4fd9-4578-851b-f0bb1cdff864"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 20:53:26.147572 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.147559    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "77092fca-4fd9-4578-851b-f0bb1cdff864" (UID: "77092fca-4fd9-4578-851b-f0bb1cdff864"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 20:53:26.148044 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.148024    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "77092fca-4fd9-4578-851b-f0bb1cdff864" (UID: "77092fca-4fd9-4578-851b-f0bb1cdff864"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 20:53:26.148669 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.148644    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77092fca-4fd9-4578-851b-f0bb1cdff864-kube-api-access-mq8hd" (OuterVolumeSpecName: "kube-api-access-mq8hd") pod "77092fca-4fd9-4578-851b-f0bb1cdff864" (UID: "77092fca-4fd9-4578-851b-f0bb1cdff864"). InnerVolumeSpecName "kube-api-access-mq8hd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 20:53:26.159718 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.159695    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-web-config" (OuterVolumeSpecName: "web-config") pod "77092fca-4fd9-4578-851b-f0bb1cdff864" (UID: "77092fca-4fd9-4578-851b-f0bb1cdff864"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 20:53:26.161521 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.161497    2567 generic.go:356] "Generic (PLEG): container finished" podID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerID="d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d" exitCode=0
Sep 17 20:53:26.161521 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.161521    2567 generic.go:356] "Generic (PLEG): container finished" podID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerID="f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1" exitCode=0
Sep 17 20:53:26.161629 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.161528    2567 generic.go:356] "Generic (PLEG): container finished" podID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerID="65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0" exitCode=0
Sep 17 20:53:26.161629 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.161534    2567 generic.go:356] "Generic (PLEG): container finished" podID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerID="37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec" exitCode=0
Sep 17 20:53:26.161629 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.161539    2567 generic.go:356] "Generic (PLEG): container finished" podID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerID="3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702" exitCode=0
Sep 17 20:53:26.161629 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.161544    2567 generic.go:356] "Generic (PLEG): container finished" podID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerID="a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3" exitCode=0
Sep 17 20:53:26.161629 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.161571    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.161629 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.161573    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"77092fca-4fd9-4578-851b-f0bb1cdff864","Type":"ContainerDied","Data":"d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d"}
Sep 17 20:53:26.161826 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.161670    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"77092fca-4fd9-4578-851b-f0bb1cdff864","Type":"ContainerDied","Data":"f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1"}
Sep 17 20:53:26.161826 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.161700    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"77092fca-4fd9-4578-851b-f0bb1cdff864","Type":"ContainerDied","Data":"65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0"}
Sep 17 20:53:26.161826 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.161724    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"77092fca-4fd9-4578-851b-f0bb1cdff864","Type":"ContainerDied","Data":"37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec"}
Sep 17 20:53:26.161826 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.161736    2567 scope.go:117] "RemoveContainer" containerID="d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d"
Sep 17 20:53:26.161826 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.161747    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"77092fca-4fd9-4578-851b-f0bb1cdff864","Type":"ContainerDied","Data":"3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702"}
Sep 17 20:53:26.161826 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.161804    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"77092fca-4fd9-4578-851b-f0bb1cdff864","Type":"ContainerDied","Data":"a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3"}
Sep 17 20:53:26.162028 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.161830    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"77092fca-4fd9-4578-851b-f0bb1cdff864","Type":"ContainerDied","Data":"15c52658ad530b7772972ef51300b37febdcadca219e89761ec07574319cbb9a"}
Sep 17 20:53:26.170787 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.170745    2567 scope.go:117] "RemoveContainer" containerID="f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1"
Sep 17 20:53:26.179453 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.179432    2567 scope.go:117] "RemoveContainer" containerID="65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0"
Sep 17 20:53:26.189816 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.188348    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 17 20:53:26.189816 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.188533    2567 scope.go:117] "RemoveContainer" containerID="37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec"
Sep 17 20:53:26.191229 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.191205    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 17 20:53:26.195834 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.195818    2567 scope.go:117] "RemoveContainer" containerID="3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702"
Sep 17 20:53:26.202400 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.202379    2567 scope.go:117] "RemoveContainer" containerID="a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3"
Sep 17 20:53:26.209186 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.209169    2567 scope.go:117] "RemoveContainer" containerID="6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98"
Sep 17 20:53:26.215529 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.215513    2567 scope.go:117] "RemoveContainer" containerID="d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d"
Sep 17 20:53:26.215869 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:53:26.215850    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d\": container with ID starting with d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d not found: ID does not exist" containerID="d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d"
Sep 17 20:53:26.215948 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.215882    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d"} err="failed to get container status \"d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d\": rpc error: code = NotFound desc = could not find container \"d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d\": container with ID starting with d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d not found: ID does not exist"
Sep 17 20:53:26.215948 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.215906    2567 scope.go:117] "RemoveContainer" containerID="f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1"
Sep 17 20:53:26.216337 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:53:26.216302    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1\": container with ID starting with f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1 not found: ID does not exist" containerID="f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1"
Sep 17 20:53:26.216429 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.216331    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1"} err="failed to get container status \"f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1\": rpc error: code = NotFound desc = could not find container \"f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1\": container with ID starting with f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1 not found: ID does not exist"
Sep 17 20:53:26.216429 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.216351    2567 scope.go:117] "RemoveContainer" containerID="65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0"
Sep 17 20:53:26.216814 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:53:26.216787    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0\": container with ID starting with 65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0 not found: ID does not exist" containerID="65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0"
Sep 17 20:53:26.216814 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.216819    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0"} err="failed to get container status \"65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0\": rpc error: code = NotFound desc = could not find container \"65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0\": container with ID starting with 65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0 not found: ID does not exist"
Sep 17 20:53:26.216814 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.216839    2567 scope.go:117] "RemoveContainer" containerID="37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec"
Sep 17 20:53:26.217150 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:53:26.217127    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec\": container with ID starting with 37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec not found: ID does not exist" containerID="37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec"
Sep 17 20:53:26.217203 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.217158    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec"} err="failed to get container status \"37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec\": rpc error: code = NotFound desc = could not find container \"37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec\": container with ID starting with 37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec not found: ID does not exist"
Sep 17 20:53:26.217203 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.217179    2567 scope.go:117] "RemoveContainer" containerID="3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702"
Sep 17 20:53:26.217480 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:53:26.217432    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702\": container with ID starting with 3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702 not found: ID does not exist" containerID="3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702"
Sep 17 20:53:26.217480 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.217460    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702"} err="failed to get container status \"3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702\": rpc error: code = NotFound desc = could not find container \"3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702\": container with ID starting with 3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702 not found: ID does not exist"
Sep 17 20:53:26.217600 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.217481    2567 scope.go:117] "RemoveContainer" containerID="a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3"
Sep 17 20:53:26.217786 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:53:26.217747    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3\": container with ID starting with a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3 not found: ID does not exist" containerID="a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3"
Sep 17 20:53:26.217830 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.217792    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3"} err="failed to get container status \"a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3\": rpc error: code = NotFound desc = could not find container \"a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3\": container with ID starting with a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3 not found: ID does not exist"
Sep 17 20:53:26.217830 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.217807    2567 scope.go:117] "RemoveContainer" containerID="6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98"
Sep 17 20:53:26.218045 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218028    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 17 20:53:26.218085 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:53:26.218044    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98\": container with ID starting with 6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98 not found: ID does not exist" containerID="6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98"
Sep 17 20:53:26.218085 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218062    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98"} err="failed to get container status \"6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98\": rpc error: code = NotFound desc = could not find container \"6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98\": container with ID starting with 6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98 not found: ID does not exist"
Sep 17 20:53:26.218085 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218075    2567 scope.go:117] "RemoveContainer" containerID="d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d"
Sep 17 20:53:26.218320 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218307    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerName="config-reloader"
Sep 17 20:53:26.218386 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218321    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerName="config-reloader"
Sep 17 20:53:26.218386 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218313    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d"} err="failed to get container status \"d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d\": rpc error: code = NotFound desc = could not find container \"d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d\": container with ID starting with d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d not found: ID does not exist"
Sep 17 20:53:26.218386 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218331    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerName="prometheus"
Sep 17 20:53:26.218386 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218337    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerName="prometheus"
Sep 17 20:53:26.218386 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218336    2567 scope.go:117] "RemoveContainer" containerID="f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1"
Sep 17 20:53:26.218386 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218345    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerName="init-config-reloader"
Sep 17 20:53:26.218386 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218351    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerName="init-config-reloader"
Sep 17 20:53:26.218386 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218361    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerName="kube-rbac-proxy-thanos"
Sep 17 20:53:26.218386 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218367    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerName="kube-rbac-proxy-thanos"
Sep 17 20:53:26.218386 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218374    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerName="kube-rbac-proxy-web"
Sep 17 20:53:26.218386 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218379    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerName="kube-rbac-proxy-web"
Sep 17 20:53:26.218386 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218387    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerName="thanos-sidecar"
Sep 17 20:53:26.218386 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218392    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerName="thanos-sidecar"
Sep 17 20:53:26.218883 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218405    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="662880e7-4dd4-4556-a2db-ba6b415784aa" containerName="registry"
Sep 17 20:53:26.218883 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218411    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="662880e7-4dd4-4556-a2db-ba6b415784aa" containerName="registry"
Sep 17 20:53:26.218883 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218420    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerName="kube-rbac-proxy"
Sep 17 20:53:26.218883 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218426    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerName="kube-rbac-proxy"
Sep 17 20:53:26.218883 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218433    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e8fb726-7627-4b4a-820f-af3caa14b16d" containerName="console"
Sep 17 20:53:26.218883 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218440    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e8fb726-7627-4b4a-820f-af3caa14b16d" containerName="console"
Sep 17 20:53:26.218883 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218492    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerName="config-reloader"
Sep 17 20:53:26.218883 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218503    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="662880e7-4dd4-4556-a2db-ba6b415784aa" containerName="registry"
Sep 17 20:53:26.218883 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218509    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerName="thanos-sidecar"
Sep 17 20:53:26.218883 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218514    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerName="kube-rbac-proxy-web"
Sep 17 20:53:26.218883 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218520    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerName="prometheus"
Sep 17 20:53:26.218883 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218526    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerName="kube-rbac-proxy"
Sep 17 20:53:26.218883 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218535    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="77092fca-4fd9-4578-851b-f0bb1cdff864" containerName="kube-rbac-proxy-thanos"
Sep 17 20:53:26.218883 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218541    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e8fb726-7627-4b4a-820f-af3caa14b16d" containerName="console"
Sep 17 20:53:26.218883 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218568    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1"} err="failed to get container status \"f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1\": rpc error: code = NotFound desc = could not find container \"f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1\": container with ID starting with f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1 not found: ID does not exist"
Sep 17 20:53:26.218883 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218582    2567 scope.go:117] "RemoveContainer" containerID="65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0"
Sep 17 20:53:26.218883 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218857    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0"} err="failed to get container status \"65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0\": rpc error: code = NotFound desc = could not find container \"65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0\": container with ID starting with 65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0 not found: ID does not exist"
Sep 17 20:53:26.218883 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.218873    2567 scope.go:117] "RemoveContainer" containerID="37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec"
Sep 17 20:53:26.219550 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.219094    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec"} err="failed to get container status \"37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec\": rpc error: code = NotFound desc = could not find container \"37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec\": container with ID starting with 37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec not found: ID does not exist"
Sep 17 20:53:26.219550 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.219111    2567 scope.go:117] "RemoveContainer" containerID="3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702"
Sep 17 20:53:26.219550 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.219345    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702"} err="failed to get container status \"3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702\": rpc error: code = NotFound desc = could not find container \"3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702\": container with ID starting with 3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702 not found: ID does not exist"
Sep 17 20:53:26.219550 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.219362    2567 scope.go:117] "RemoveContainer" containerID="a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3"
Sep 17 20:53:26.219550 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.219526    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3"} err="failed to get container status \"a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3\": rpc error: code = NotFound desc = could not find container \"a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3\": container with ID starting with a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3 not found: ID does not exist"
Sep 17 20:53:26.219550 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.219538    2567 scope.go:117] "RemoveContainer" containerID="6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98"
Sep 17 20:53:26.219793 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.219770    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98"} err="failed to get container status \"6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98\": rpc error: code = NotFound desc = could not find container \"6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98\": container with ID starting with 6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98 not found: ID does not exist"
Sep 17 20:53:26.219837 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.219794    2567 scope.go:117] "RemoveContainer" containerID="d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d"
Sep 17 20:53:26.220032 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.220015    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d"} err="failed to get container status \"d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d\": rpc error: code = NotFound desc = could not find container \"d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d\": container with ID starting with d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d not found: ID does not exist"
Sep 17 20:53:26.220032 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.220030    2567 scope.go:117] "RemoveContainer" containerID="f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1"
Sep 17 20:53:26.220251 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.220237    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1"} err="failed to get container status \"f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1\": rpc error: code = NotFound desc = could not find container \"f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1\": container with ID starting with f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1 not found: ID does not exist"
Sep 17 20:53:26.220296 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.220252    2567 scope.go:117] "RemoveContainer" containerID="65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0"
Sep 17 20:53:26.220469 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.220450    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0"} err="failed to get container status \"65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0\": rpc error: code = NotFound desc = could not find container \"65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0\": container with ID starting with 65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0 not found: ID does not exist"
Sep 17 20:53:26.220514 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.220469    2567 scope.go:117] "RemoveContainer" containerID="37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec"
Sep 17 20:53:26.220702 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.220685    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec"} err="failed to get container status \"37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec\": rpc error: code = NotFound desc = could not find container \"37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec\": container with ID starting with 37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec not found: ID does not exist"
Sep 17 20:53:26.220702 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.220702    2567 scope.go:117] "RemoveContainer" containerID="3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702"
Sep 17 20:53:26.220928 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.220909    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702"} err="failed to get container status \"3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702\": rpc error: code = NotFound desc = could not find container \"3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702\": container with ID starting with 3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702 not found: ID does not exist"
Sep 17 20:53:26.220982 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.220929    2567 scope.go:117] "RemoveContainer" containerID="a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3"
Sep 17 20:53:26.221163 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.221147    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3"} err="failed to get container status \"a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3\": rpc error: code = NotFound desc = could not find container \"a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3\": container with ID starting with a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3 not found: ID does not exist"
Sep 17 20:53:26.221163 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.221163    2567 scope.go:117] "RemoveContainer" containerID="6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98"
Sep 17 20:53:26.221379 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.221362    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98"} err="failed to get container status \"6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98\": rpc error: code = NotFound desc = could not find container \"6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98\": container with ID starting with 6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98 not found: ID does not exist"
Sep 17 20:53:26.221423 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.221380    2567 scope.go:117] "RemoveContainer" containerID="d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d"
Sep 17 20:53:26.221606 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.221589    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d"} err="failed to get container status \"d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d\": rpc error: code = NotFound desc = could not find container \"d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d\": container with ID starting with d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d not found: ID does not exist"
Sep 17 20:53:26.221606 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.221605    2567 scope.go:117] "RemoveContainer" containerID="f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1"
Sep 17 20:53:26.221842 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.221824    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1"} err="failed to get container status \"f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1\": rpc error: code = NotFound desc = could not find container \"f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1\": container with ID starting with f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1 not found: ID does not exist"
Sep 17 20:53:26.221842 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.221840    2567 scope.go:117] "RemoveContainer" containerID="65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0"
Sep 17 20:53:26.222072 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.222049    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0"} err="failed to get container status \"65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0\": rpc error: code = NotFound desc = could not find container \"65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0\": container with ID starting with 65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0 not found: ID does not exist"
Sep 17 20:53:26.222164 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.222112    2567 scope.go:117] "RemoveContainer" containerID="37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec"
Sep 17 20:53:26.222450 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.222426    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec"} err="failed to get container status \"37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec\": rpc error: code = NotFound desc = could not find container \"37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec\": container with ID starting with 37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec not found: ID does not exist"
Sep 17 20:53:26.222450 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.222451    2567 scope.go:117] "RemoveContainer" containerID="3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702"
Sep 17 20:53:26.222798 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.222748    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702"} err="failed to get container status \"3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702\": rpc error: code = NotFound desc = could not find container \"3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702\": container with ID starting with 3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702 not found: ID does not exist"
Sep 17 20:53:26.222877 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.222800    2567 scope.go:117] "RemoveContainer" containerID="a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3"
Sep 17 20:53:26.223052 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.223032    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3"} err="failed to get container status \"a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3\": rpc error: code = NotFound desc = could not find container \"a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3\": container with ID starting with a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3 not found: ID does not exist"
Sep 17 20:53:26.223127 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.223053    2567 scope.go:117] "RemoveContainer" containerID="6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98"
Sep 17 20:53:26.223329 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.223311    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98"} err="failed to get container status \"6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98\": rpc error: code = NotFound desc = could not find container \"6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98\": container with ID starting with 6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98 not found: ID does not exist"
Sep 17 20:53:26.223384 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.223329    2567 scope.go:117] "RemoveContainer" containerID="d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d"
Sep 17 20:53:26.223558 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.223540    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d"} err="failed to get container status \"d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d\": rpc error: code = NotFound desc = could not find container \"d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d\": container with ID starting with d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d not found: ID does not exist"
Sep 17 20:53:26.223608 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.223558    2567 scope.go:117] "RemoveContainer" containerID="f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1"
Sep 17 20:53:26.223832 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.223811    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1"} err="failed to get container status \"f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1\": rpc error: code = NotFound desc = could not find container \"f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1\": container with ID starting with f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1 not found: ID does not exist"
Sep 17 20:53:26.223832 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.223830    2567 scope.go:117] "RemoveContainer" containerID="65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0"
Sep 17 20:53:26.223964 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.223947    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.224060 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.224042    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0"} err="failed to get container status \"65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0\": rpc error: code = NotFound desc = could not find container \"65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0\": container with ID starting with 65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0 not found: ID does not exist"
Sep 17 20:53:26.224116 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.224062    2567 scope.go:117] "RemoveContainer" containerID="37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec"
Sep 17 20:53:26.224380 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.224356    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec"} err="failed to get container status \"37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec\": rpc error: code = NotFound desc = could not find container \"37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec\": container with ID starting with 37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec not found: ID does not exist"
Sep 17 20:53:26.224481 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.224382    2567 scope.go:117] "RemoveContainer" containerID="3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702"
Sep 17 20:53:26.224660 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.224635    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702"} err="failed to get container status \"3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702\": rpc error: code = NotFound desc = could not find container \"3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702\": container with ID starting with 3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702 not found: ID does not exist"
Sep 17 20:53:26.224727 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.224663    2567 scope.go:117] "RemoveContainer" containerID="a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3"
Sep 17 20:53:26.224943 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.224923    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3"} err="failed to get container status \"a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3\": rpc error: code = NotFound desc = could not find container \"a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3\": container with ID starting with a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3 not found: ID does not exist"
Sep 17 20:53:26.224993 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.224945    2567 scope.go:117] "RemoveContainer" containerID="6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98"
Sep 17 20:53:26.225212 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.225195    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98"} err="failed to get container status \"6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98\": rpc error: code = NotFound desc = could not find container \"6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98\": container with ID starting with 6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98 not found: ID does not exist"
Sep 17 20:53:26.225253 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.225214    2567 scope.go:117] "RemoveContainer" containerID="d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d"
Sep 17 20:53:26.225464 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.225445    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d"} err="failed to get container status \"d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d\": rpc error: code = NotFound desc = could not find container \"d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d\": container with ID starting with d4351867013b4a9f51bcaf4ad9c23bbda1cbfea8dec7c3d7e731725e2692d85d not found: ID does not exist"
Sep 17 20:53:26.225507 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.225464    2567 scope.go:117] "RemoveContainer" containerID="f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1"
Sep 17 20:53:26.225737 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.225716    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1"} err="failed to get container status \"f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1\": rpc error: code = NotFound desc = could not find container \"f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1\": container with ID starting with f4b68e47ef0a504b16295ff998e1f58354861bbb84cddb078abd0cbf9c627eb1 not found: ID does not exist"
Sep 17 20:53:26.225808 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.225738    2567 scope.go:117] "RemoveContainer" containerID="65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0"
Sep 17 20:53:26.226074 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.226056    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0"} err="failed to get container status \"65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0\": rpc error: code = NotFound desc = could not find container \"65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0\": container with ID starting with 65f7dcab8c699c165d5cf55bc7dc57fae34f18a0f50cb6a72aba39eafa7c0eb0 not found: ID does not exist"
Sep 17 20:53:26.226117 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.226075    2567 scope.go:117] "RemoveContainer" containerID="37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec"
Sep 17 20:53:26.226311 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.226291    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec"} err="failed to get container status \"37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec\": rpc error: code = NotFound desc = could not find container \"37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec\": container with ID starting with 37b6e9aeaeb43610fbd741eb013ca339fc00a7d57ae1ae23484b700a6c2be5ec not found: ID does not exist"
Sep 17 20:53:26.226373 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.226312    2567 scope.go:117] "RemoveContainer" containerID="3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702"
Sep 17 20:53:26.226532 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.226512    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 17 20:53:26.226612 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.226588    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702"} err="failed to get container status \"3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702\": rpc error: code = NotFound desc = could not find container \"3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702\": container with ID starting with 3c58992b66ac4ecbe23fc00fc59fa5493856ace4a628211fd947b7e2780e1702 not found: ID does not exist"
Sep 17 20:53:26.226688 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.226613    2567 scope.go:117] "RemoveContainer" containerID="a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3"
Sep 17 20:53:26.226857 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.226841    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 17 20:53:26.226919 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.226895    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3"} err="failed to get container status \"a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3\": rpc error: code = NotFound desc = could not find container \"a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3\": container with ID starting with a1227ec10bf01e059f6be4f3b94febe119bc23e8c767de95342ab903d08932b3 not found: ID does not exist"
Sep 17 20:53:26.226919 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.226907    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 17 20:53:26.226919 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.226846    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-hw2fb\""
Sep 17 20:53:26.227069 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.226914    2567 scope.go:117] "RemoveContainer" containerID="6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98"
Sep 17 20:53:26.227150 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.227134    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 17 20:53:26.227244 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.227230    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 17 20:53:26.227295 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.227242    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 17 20:53:26.227343 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.227305    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 17 20:53:26.227426 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.227402    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98"} err="failed to get container status \"6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98\": rpc error: code = NotFound desc = could not find container \"6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98\": container with ID starting with 6c297e22620062fb158ff1c3dd6bae7f38b092f34604b32dc6056f0e1b88eb98 not found: ID does not exist"
Sep 17 20:53:26.227489 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.227428    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 17 20:53:26.227489 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.227465    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 17 20:53:26.227489 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.227483    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 17 20:53:26.227687 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.227483    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 17 20:53:26.227687 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.227572    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-e55qh19ij7uk\""
Sep 17 20:53:26.227808 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.227792    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 17 20:53:26.228131 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.228115    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 17 20:53:26.230632 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.230613    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 17 20:53:26.232503 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.232484    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 17 20:53:26.240130 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.238991    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 17 20:53:26.243312 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.243293    2567 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/77092fca-4fd9-4578-851b-f0bb1cdff864-config-out\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:53:26.243392 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.243329    2567 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-web-config\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:53:26.243392 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.243339    2567 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/77092fca-4fd9-4578-851b-f0bb1cdff864-tls-assets\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:53:26.243392 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.243348    2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/77092fca-4fd9-4578-851b-f0bb1cdff864-prometheus-k8s-db\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:53:26.243392 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.243357    2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:53:26.243392 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.243366    2567 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:53:26.243392 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.243375    2567 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-kube-rbac-proxy\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:53:26.243392 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.243383    2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:53:26.243392 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.243391    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mq8hd\" (UniqueName: \"kubernetes.io/projected/77092fca-4fd9-4578-851b-f0bb1cdff864-kube-api-access-mq8hd\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:53:26.243622 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.243400    2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-prometheus-k8s-tls\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:53:26.243622 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.243408    2567 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-metrics-client-certs\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:53:26.243622 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.243416    2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:53:26.243622 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.243426    2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-prometheus-trusted-ca-bundle\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:53:26.243622 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.243434    2567 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-grpc-tls\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:53:26.243622 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.243442    2567 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-configmap-metrics-client-ca\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:53:26.243622 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.243450    2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/77092fca-4fd9-4578-851b-f0bb1cdff864-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:53:26.243622 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.243458    2567 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-config\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:53:26.243622 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.243466    2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:53:26.243622 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.243475    2567 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/77092fca-4fd9-4578-851b-f0bb1cdff864-thanos-prometheus-http-client-file\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:53:26.344177 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.344137    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/daccfc05-4529-4276-b683-78d5982cd20f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.344177 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.344176    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/daccfc05-4529-4276-b683-78d5982cd20f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.344380 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.344206    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/daccfc05-4529-4276-b683-78d5982cd20f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.344380 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.344259    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/daccfc05-4529-4276-b683-78d5982cd20f-web-config\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.344380 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.344308    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/daccfc05-4529-4276-b683-78d5982cd20f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.344380 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.344335    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/daccfc05-4529-4276-b683-78d5982cd20f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.344380 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.344355    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/daccfc05-4529-4276-b683-78d5982cd20f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.344380 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.344372    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/daccfc05-4529-4276-b683-78d5982cd20f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.344564 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.344402    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/daccfc05-4529-4276-b683-78d5982cd20f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.344564 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.344424    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/daccfc05-4529-4276-b683-78d5982cd20f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.344564 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.344461    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/daccfc05-4529-4276-b683-78d5982cd20f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.344564 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.344477    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/daccfc05-4529-4276-b683-78d5982cd20f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.344564 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.344496    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqlvm\" (UniqueName: \"kubernetes.io/projected/daccfc05-4529-4276-b683-78d5982cd20f-kube-api-access-gqlvm\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.344564 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.344520    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/daccfc05-4529-4276-b683-78d5982cd20f-config-out\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.344564 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.344538    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/daccfc05-4529-4276-b683-78d5982cd20f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.344564 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.344554    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/daccfc05-4529-4276-b683-78d5982cd20f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.344880 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.344569    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/daccfc05-4529-4276-b683-78d5982cd20f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.344880 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.344593    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/daccfc05-4529-4276-b683-78d5982cd20f-config\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.344880 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.344609    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/daccfc05-4529-4276-b683-78d5982cd20f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.344880 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.344631    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/daccfc05-4529-4276-b683-78d5982cd20f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.445120 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.445031    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/daccfc05-4529-4276-b683-78d5982cd20f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.445120 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.445078    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/daccfc05-4529-4276-b683-78d5982cd20f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.445329 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.445195    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/daccfc05-4529-4276-b683-78d5982cd20f-web-config\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.445329 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.445229    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/daccfc05-4529-4276-b683-78d5982cd20f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.445329 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.445253    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/daccfc05-4529-4276-b683-78d5982cd20f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.445329 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.445274    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/daccfc05-4529-4276-b683-78d5982cd20f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.445329 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.445298    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/daccfc05-4529-4276-b683-78d5982cd20f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.445574 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.445361    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/daccfc05-4529-4276-b683-78d5982cd20f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.445574 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.445399    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/daccfc05-4529-4276-b683-78d5982cd20f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.445574 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.445434    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/daccfc05-4529-4276-b683-78d5982cd20f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.445574 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.445459    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/daccfc05-4529-4276-b683-78d5982cd20f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.445574 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.445493    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqlvm\" (UniqueName: \"kubernetes.io/projected/daccfc05-4529-4276-b683-78d5982cd20f-kube-api-access-gqlvm\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.445574 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.445518    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/daccfc05-4529-4276-b683-78d5982cd20f-config-out\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.445574 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.445536    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/daccfc05-4529-4276-b683-78d5982cd20f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.445941 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.445724    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/daccfc05-4529-4276-b683-78d5982cd20f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.445941 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.445540    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/daccfc05-4529-4276-b683-78d5982cd20f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.445941 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.445819    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/daccfc05-4529-4276-b683-78d5982cd20f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.445941 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.445852    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/daccfc05-4529-4276-b683-78d5982cd20f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.445941 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.445882    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/daccfc05-4529-4276-b683-78d5982cd20f-config\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.445941 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.445905    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/daccfc05-4529-4276-b683-78d5982cd20f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.446244 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.445943    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/daccfc05-4529-4276-b683-78d5982cd20f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.446244 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.445981    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/daccfc05-4529-4276-b683-78d5982cd20f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.446355 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.446334    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/daccfc05-4529-4276-b683-78d5982cd20f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.446959 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.446930    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/daccfc05-4529-4276-b683-78d5982cd20f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.449323 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.447559    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/daccfc05-4529-4276-b683-78d5982cd20f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.449323 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.447830    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/daccfc05-4529-4276-b683-78d5982cd20f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.449323 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.448169    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/daccfc05-4529-4276-b683-78d5982cd20f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.449323 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.448465    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/daccfc05-4529-4276-b683-78d5982cd20f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.449323 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.448526    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/daccfc05-4529-4276-b683-78d5982cd20f-web-config\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.449323 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.449017    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/daccfc05-4529-4276-b683-78d5982cd20f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.449323 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.449093    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/daccfc05-4529-4276-b683-78d5982cd20f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.449323 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.449306    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/daccfc05-4529-4276-b683-78d5982cd20f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.450034 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.449846    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/daccfc05-4529-4276-b683-78d5982cd20f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.450034 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.449992    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/daccfc05-4529-4276-b683-78d5982cd20f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.450148 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.450076    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/daccfc05-4529-4276-b683-78d5982cd20f-config\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.450521 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.450499    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/daccfc05-4529-4276-b683-78d5982cd20f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.450876 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.450850    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/daccfc05-4529-4276-b683-78d5982cd20f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.451008 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.450987    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/daccfc05-4529-4276-b683-78d5982cd20f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.451044 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.451000    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/daccfc05-4529-4276-b683-78d5982cd20f-config-out\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.458005 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.457984    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqlvm\" (UniqueName: \"kubernetes.io/projected/daccfc05-4529-4276-b683-78d5982cd20f-kube-api-access-gqlvm\") pod \"prometheus-k8s-0\" (UID: \"daccfc05-4529-4276-b683-78d5982cd20f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.534160 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.534123    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:26.817916 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:26.817892    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 17 20:53:26.819973 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:53:26.819940    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaccfc05_4529_4276_b683_78d5982cd20f.slice/crio-2b0f784b86805c26924ecc4582dddbde624fb0ff0553eef45e37fffcd47fe8a1 WatchSource:0}: Error finding container 2b0f784b86805c26924ecc4582dddbde624fb0ff0553eef45e37fffcd47fe8a1: Status 404 returned error can't find the container with id 2b0f784b86805c26924ecc4582dddbde624fb0ff0553eef45e37fffcd47fe8a1
Sep 17 20:53:27.167890 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:27.167803    2567 generic.go:356] "Generic (PLEG): container finished" podID="daccfc05-4529-4276-b683-78d5982cd20f" containerID="79f4464bd84b40dea3f725f408e1e6a7adf51d1e9ca57416d3f20f57ea436a41" exitCode=0
Sep 17 20:53:27.167890 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:27.167851    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"daccfc05-4529-4276-b683-78d5982cd20f","Type":"ContainerDied","Data":"79f4464bd84b40dea3f725f408e1e6a7adf51d1e9ca57416d3f20f57ea436a41"}
Sep 17 20:53:27.168112 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:27.167897    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"daccfc05-4529-4276-b683-78d5982cd20f","Type":"ContainerStarted","Data":"2b0f784b86805c26924ecc4582dddbde624fb0ff0553eef45e37fffcd47fe8a1"}
Sep 17 20:53:28.169041 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:28.168960    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77092fca-4fd9-4578-851b-f0bb1cdff864" path="/var/lib/kubelet/pods/77092fca-4fd9-4578-851b-f0bb1cdff864/volumes"
Sep 17 20:53:28.177420 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:28.177393    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"daccfc05-4529-4276-b683-78d5982cd20f","Type":"ContainerStarted","Data":"7d5555ec6e621a19a92e9549f926cb93cd6057faf6b530c67b1a1c1db3f3372c"}
Sep 17 20:53:28.177553 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:28.177449    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"daccfc05-4529-4276-b683-78d5982cd20f","Type":"ContainerStarted","Data":"fa2004d4cdaa14b04170958d90c8e6d8aa67e8acab3247d94ced5f336386f2a5"}
Sep 17 20:53:28.177553 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:28.177481    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"daccfc05-4529-4276-b683-78d5982cd20f","Type":"ContainerStarted","Data":"cafe306d7908776f95a30c44e6f7605f828c3004ebaa3f2f842a442ed9ffd9e2"}
Sep 17 20:53:28.177553 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:28.177505    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"daccfc05-4529-4276-b683-78d5982cd20f","Type":"ContainerStarted","Data":"3f81bad65ab8668e21aa944a2cb9b8cca4e56b7775de0e3c3c02f07d46fdb101"}
Sep 17 20:53:28.177553 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:28.177530    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"daccfc05-4529-4276-b683-78d5982cd20f","Type":"ContainerStarted","Data":"245e74371b9865d6545e7eb9771e60bca811732131f03623f120d4ed20c62ee8"}
Sep 17 20:53:28.177553 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:28.177551    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"daccfc05-4529-4276-b683-78d5982cd20f","Type":"ContainerStarted","Data":"fede195a0ce5523232e423aa844c05f2c65d4a9adb15d8fe2e4f69acb30d0fb8"}
Sep 17 20:53:28.212268 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:28.212218    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.194440632 podStartE2EDuration="2.2122017s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.761068ms" isStatefulPod=false podCreationTimestamp="2026-09-17 20:53:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:53:28.20969211 +0000 UTC m=+272.555293932" watchObservedRunningTime="2026-09-17 20:53:28.2122017 +0000 UTC m=+272.557803581"
Sep 17 20:53:28.582149 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:28.582114    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:53:28.679026 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:28.678981    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:53:31.534710 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:31.534667    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:53:34.523724 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:34.523688    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:53:34.587584 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:34.587558    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:53:35.239637 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:35.239600    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:53:35.304657 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:35.304627    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:53:39.579200 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:39.579161    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:53:39.681619 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:39.681583    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:53:39.739092 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:39.739061    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:53:56.085268 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:53:56.085227    2567 kubelet.go:1648] "Image garbage collection succeeded"
Sep 17 20:54:16.358336 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:54:16.358303    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:54:26.534960 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:54:26.534920    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:54:26.550111 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:54:26.550085    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:54:27.409592 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:54:27.409565    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 17 20:54:35.645225 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:54:35.645189    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:54:35.718714 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:54:35.718682    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:54:37.152275 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:54:37.152244    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:54:37.249404 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:54:37.249372    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:54:47.654455 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:54:47.654385    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:54:47.729351 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:54:47.729314    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:54:47.833542 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:54:47.833496    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:54:47.934568 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:54:47.934482    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:54:47.986916 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:54:47.986886    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:54:58.762594 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:54:58.762469    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:54:58.862940 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:54:58.862905    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:55:00.481711 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:55:00.481670    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:55:00.578898 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:55:00.578858    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:55:05.783099 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:55:05.783056    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:55:14.701591 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:55:14.701547    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:55:14.764639 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:55:14.764600    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:55:42.248932 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:55:42.248884    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:55:42.344868 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:55:42.344828    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:55:42.437504 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:55:42.437472    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:56:01.300516 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:01.300471    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:56:01.402271 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:01.402231    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:56:01.507180 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:01.507140    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:56:24.144989 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:24.144957    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:56:25.577283 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:25.577250    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:56:30.130743 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:30.130700    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:56:30.498556 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:30.498516    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:56:30.715590 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:30.715548    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:56:30.871658 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:30.871571    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:56:30.970361 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:30.970332    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:56:31.100669 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:31.100631    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:56:31.243718 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:31.243648    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:56:31.347478 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:31.347443    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:56:31.595744 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:31.595665    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:56:31.804057 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:31.804026    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:56:31.962302 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:31.962272    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:56:32.069388 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:32.069337    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:56:48.734464 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:48.734424    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:56:57.286052 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.286020    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-864d48b4c4-xcq5p"]
Sep 17 20:56:57.289067 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.289051    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-864d48b4c4-xcq5p"
Sep 17 20:56:57.293474 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.293450    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 17 20:56:57.294423 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.294401    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 17 20:56:57.294618 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.294522    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 17 20:56:57.294618 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.294430    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 17 20:56:57.294618 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.294590    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-7mwmf\""
Sep 17 20:56:57.294851 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.294465    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 17 20:56:57.294851 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.294424    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 17 20:56:57.295133 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.295119    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 17 20:56:57.298169 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.298146    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 17 20:56:57.303274 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.303252    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-864d48b4c4-xcq5p"]
Sep 17 20:56:57.344992 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.344958    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:56:57.351853 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.351829    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e5b8b8df-1654-4edc-87f5-db9306ddfc52-console-config\") pod \"console-864d48b4c4-xcq5p\" (UID: \"e5b8b8df-1654-4edc-87f5-db9306ddfc52\") " pod="openshift-console/console-864d48b4c4-xcq5p"
Sep 17 20:56:57.351949 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.351876    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e5b8b8df-1654-4edc-87f5-db9306ddfc52-console-serving-cert\") pod \"console-864d48b4c4-xcq5p\" (UID: \"e5b8b8df-1654-4edc-87f5-db9306ddfc52\") " pod="openshift-console/console-864d48b4c4-xcq5p"
Sep 17 20:56:57.351949 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.351917    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e5b8b8df-1654-4edc-87f5-db9306ddfc52-console-oauth-config\") pod \"console-864d48b4c4-xcq5p\" (UID: \"e5b8b8df-1654-4edc-87f5-db9306ddfc52\") " pod="openshift-console/console-864d48b4c4-xcq5p"
Sep 17 20:56:57.352037 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.351950    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd84c\" (UniqueName: \"kubernetes.io/projected/e5b8b8df-1654-4edc-87f5-db9306ddfc52-kube-api-access-xd84c\") pod \"console-864d48b4c4-xcq5p\" (UID: \"e5b8b8df-1654-4edc-87f5-db9306ddfc52\") " pod="openshift-console/console-864d48b4c4-xcq5p"
Sep 17 20:56:57.352037 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.351975    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e5b8b8df-1654-4edc-87f5-db9306ddfc52-service-ca\") pod \"console-864d48b4c4-xcq5p\" (UID: \"e5b8b8df-1654-4edc-87f5-db9306ddfc52\") " pod="openshift-console/console-864d48b4c4-xcq5p"
Sep 17 20:56:57.352037 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.352008    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5b8b8df-1654-4edc-87f5-db9306ddfc52-trusted-ca-bundle\") pod \"console-864d48b4c4-xcq5p\" (UID: \"e5b8b8df-1654-4edc-87f5-db9306ddfc52\") " pod="openshift-console/console-864d48b4c4-xcq5p"
Sep 17 20:56:57.352143 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.352040    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e5b8b8df-1654-4edc-87f5-db9306ddfc52-oauth-serving-cert\") pod \"console-864d48b4c4-xcq5p\" (UID: \"e5b8b8df-1654-4edc-87f5-db9306ddfc52\") " pod="openshift-console/console-864d48b4c4-xcq5p"
Sep 17 20:56:57.405695 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.405668    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:56:57.452640 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.452609    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e5b8b8df-1654-4edc-87f5-db9306ddfc52-service-ca\") pod \"console-864d48b4c4-xcq5p\" (UID: \"e5b8b8df-1654-4edc-87f5-db9306ddfc52\") " pod="openshift-console/console-864d48b4c4-xcq5p"
Sep 17 20:56:57.452817 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.452663    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5b8b8df-1654-4edc-87f5-db9306ddfc52-trusted-ca-bundle\") pod \"console-864d48b4c4-xcq5p\" (UID: \"e5b8b8df-1654-4edc-87f5-db9306ddfc52\") " pod="openshift-console/console-864d48b4c4-xcq5p"
Sep 17 20:56:57.452817 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.452700    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e5b8b8df-1654-4edc-87f5-db9306ddfc52-oauth-serving-cert\") pod \"console-864d48b4c4-xcq5p\" (UID: \"e5b8b8df-1654-4edc-87f5-db9306ddfc52\") " pod="openshift-console/console-864d48b4c4-xcq5p"
Sep 17 20:56:57.452817 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.452729    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e5b8b8df-1654-4edc-87f5-db9306ddfc52-console-config\") pod \"console-864d48b4c4-xcq5p\" (UID: \"e5b8b8df-1654-4edc-87f5-db9306ddfc52\") " pod="openshift-console/console-864d48b4c4-xcq5p"
Sep 17 20:56:57.452817 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.452776    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e5b8b8df-1654-4edc-87f5-db9306ddfc52-console-serving-cert\") pod \"console-864d48b4c4-xcq5p\" (UID: \"e5b8b8df-1654-4edc-87f5-db9306ddfc52\") " pod="openshift-console/console-864d48b4c4-xcq5p"
Sep 17 20:56:57.452817 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.452800    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e5b8b8df-1654-4edc-87f5-db9306ddfc52-console-oauth-config\") pod \"console-864d48b4c4-xcq5p\" (UID: \"e5b8b8df-1654-4edc-87f5-db9306ddfc52\") " pod="openshift-console/console-864d48b4c4-xcq5p"
Sep 17 20:56:57.453096 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.452828    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd84c\" (UniqueName: \"kubernetes.io/projected/e5b8b8df-1654-4edc-87f5-db9306ddfc52-kube-api-access-xd84c\") pod \"console-864d48b4c4-xcq5p\" (UID: \"e5b8b8df-1654-4edc-87f5-db9306ddfc52\") " pod="openshift-console/console-864d48b4c4-xcq5p"
Sep 17 20:56:57.453437 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.453409    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e5b8b8df-1654-4edc-87f5-db9306ddfc52-service-ca\") pod \"console-864d48b4c4-xcq5p\" (UID: \"e5b8b8df-1654-4edc-87f5-db9306ddfc52\") " pod="openshift-console/console-864d48b4c4-xcq5p"
Sep 17 20:56:57.453548 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.453415    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e5b8b8df-1654-4edc-87f5-db9306ddfc52-oauth-serving-cert\") pod \"console-864d48b4c4-xcq5p\" (UID: \"e5b8b8df-1654-4edc-87f5-db9306ddfc52\") " pod="openshift-console/console-864d48b4c4-xcq5p"
Sep 17 20:56:57.453695 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.453670    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5b8b8df-1654-4edc-87f5-db9306ddfc52-trusted-ca-bundle\") pod \"console-864d48b4c4-xcq5p\" (UID: \"e5b8b8df-1654-4edc-87f5-db9306ddfc52\") " pod="openshift-console/console-864d48b4c4-xcq5p"
Sep 17 20:56:57.453887 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.453870    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e5b8b8df-1654-4edc-87f5-db9306ddfc52-console-config\") pod \"console-864d48b4c4-xcq5p\" (UID: \"e5b8b8df-1654-4edc-87f5-db9306ddfc52\") " pod="openshift-console/console-864d48b4c4-xcq5p"
Sep 17 20:56:57.455290 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.455265    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e5b8b8df-1654-4edc-87f5-db9306ddfc52-console-serving-cert\") pod \"console-864d48b4c4-xcq5p\" (UID: \"e5b8b8df-1654-4edc-87f5-db9306ddfc52\") " pod="openshift-console/console-864d48b4c4-xcq5p"
Sep 17 20:56:57.455512 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.455489    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e5b8b8df-1654-4edc-87f5-db9306ddfc52-console-oauth-config\") pod \"console-864d48b4c4-xcq5p\" (UID: \"e5b8b8df-1654-4edc-87f5-db9306ddfc52\") " pod="openshift-console/console-864d48b4c4-xcq5p"
Sep 17 20:56:57.461362 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.461340    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd84c\" (UniqueName: \"kubernetes.io/projected/e5b8b8df-1654-4edc-87f5-db9306ddfc52-kube-api-access-xd84c\") pod \"console-864d48b4c4-xcq5p\" (UID: \"e5b8b8df-1654-4edc-87f5-db9306ddfc52\") " pod="openshift-console/console-864d48b4c4-xcq5p"
Sep 17 20:56:57.600365 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.600283    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-864d48b4c4-xcq5p"
Sep 17 20:56:57.870148 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.870042    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-864d48b4c4-xcq5p"]
Sep 17 20:56:57.873827 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:56:57.873798    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5b8b8df_1654_4edc_87f5_db9306ddfc52.slice/crio-d4bbb130898b815eaec4502ddfbc68ed84cd82cbb03cda1da8c05bee3d775747 WatchSource:0}: Error finding container d4bbb130898b815eaec4502ddfbc68ed84cd82cbb03cda1da8c05bee3d775747: Status 404 returned error can't find the container with id d4bbb130898b815eaec4502ddfbc68ed84cd82cbb03cda1da8c05bee3d775747
Sep 17 20:56:57.966371 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.966334    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-864d48b4c4-xcq5p" event={"ID":"e5b8b8df-1654-4edc-87f5-db9306ddfc52","Type":"ContainerStarted","Data":"e0faea6492b4bffcb85d766b56323bf5e7ab5088e011840645325b892d34a6d2"}
Sep 17 20:56:57.966488 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.966388    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-864d48b4c4-xcq5p" event={"ID":"e5b8b8df-1654-4edc-87f5-db9306ddfc52","Type":"ContainerStarted","Data":"d4bbb130898b815eaec4502ddfbc68ed84cd82cbb03cda1da8c05bee3d775747"}
Sep 17 20:56:57.985057 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:56:57.984996    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-864d48b4c4-xcq5p" podStartSLOduration=0.984975538 podStartE2EDuration="984.975538ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:56:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:56:57.983744405 +0000 UTC m=+482.329346230" watchObservedRunningTime="2026-09-17 20:56:57.984975538 +0000 UTC m=+482.330577366"
Sep 17 20:57:07.601198 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:57:07.601151    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-864d48b4c4-xcq5p"
Sep 17 20:57:07.601586 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:57:07.601229    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-864d48b4c4-xcq5p"
Sep 17 20:57:07.605990 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:57:07.605967    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-864d48b4c4-xcq5p"
Sep 17 20:57:08.009200 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:57:08.009171    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-864d48b4c4-xcq5p"
Sep 17 20:57:08.158652 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:57:08.158624    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:57:08.215443 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:57:08.215410    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:57:08.274163 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:57:08.274059    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:57:22.236445 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:57:22.236413    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:57:22.404103 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:57:22.404070    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:58:00.103222 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:00.103138    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:58:00.225403 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:00.225371    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:58:00.302748 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:00.302714    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-66f4b8957f-zccxh"]
Sep 17 20:58:00.306138 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:00.306121    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-66f4b8957f-zccxh"
Sep 17 20:58:00.311744 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:00.311717    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Sep 17 20:58:00.311899 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:00.311752    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Sep 17 20:58:00.311967 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:00.311911    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Sep 17 20:58:00.312030 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:00.311978    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-8p4d9\""
Sep 17 20:58:00.341978 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:00.341939    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-66f4b8957f-zccxh"]
Sep 17 20:58:00.353772 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:00.353693    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6l64\" (UniqueName: \"kubernetes.io/projected/fd7cfb42-7e4f-4388-9570-67e726c60a81-kube-api-access-p6l64\") pod \"kserve-controller-manager-66f4b8957f-zccxh\" (UID: \"fd7cfb42-7e4f-4388-9570-67e726c60a81\") " pod="kserve/kserve-controller-manager-66f4b8957f-zccxh"
Sep 17 20:58:00.353884 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:00.353771    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd7cfb42-7e4f-4388-9570-67e726c60a81-cert\") pod \"kserve-controller-manager-66f4b8957f-zccxh\" (UID: \"fd7cfb42-7e4f-4388-9570-67e726c60a81\") " pod="kserve/kserve-controller-manager-66f4b8957f-zccxh"
Sep 17 20:58:00.438586 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:00.438548    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:58:00.454398 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:00.454371    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6l64\" (UniqueName: \"kubernetes.io/projected/fd7cfb42-7e4f-4388-9570-67e726c60a81-kube-api-access-p6l64\") pod \"kserve-controller-manager-66f4b8957f-zccxh\" (UID: \"fd7cfb42-7e4f-4388-9570-67e726c60a81\") " pod="kserve/kserve-controller-manager-66f4b8957f-zccxh"
Sep 17 20:58:00.454525 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:00.454412    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd7cfb42-7e4f-4388-9570-67e726c60a81-cert\") pod \"kserve-controller-manager-66f4b8957f-zccxh\" (UID: \"fd7cfb42-7e4f-4388-9570-67e726c60a81\") " pod="kserve/kserve-controller-manager-66f4b8957f-zccxh"
Sep 17 20:58:00.454573 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:58:00.454540    2567 secret.go:189] Couldn't get secret kserve/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 17 20:58:00.454617 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:58:00.454607    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd7cfb42-7e4f-4388-9570-67e726c60a81-cert podName:fd7cfb42-7e4f-4388-9570-67e726c60a81 nodeName:}" failed. No retries permitted until 2026-09-17 20:58:00.95458783 +0000 UTC m=+545.300189634 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/fd7cfb42-7e4f-4388-9570-67e726c60a81-cert") pod "kserve-controller-manager-66f4b8957f-zccxh" (UID: "fd7cfb42-7e4f-4388-9570-67e726c60a81") : secret "kserve-webhook-server-cert" not found
Sep 17 20:58:00.472184 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:00.472154    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6l64\" (UniqueName: \"kubernetes.io/projected/fd7cfb42-7e4f-4388-9570-67e726c60a81-kube-api-access-p6l64\") pod \"kserve-controller-manager-66f4b8957f-zccxh\" (UID: \"fd7cfb42-7e4f-4388-9570-67e726c60a81\") " pod="kserve/kserve-controller-manager-66f4b8957f-zccxh"
Sep 17 20:58:00.541570 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:00.541537    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:58:00.657538 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:00.657442    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:58:00.958411 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:00.958387    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd7cfb42-7e4f-4388-9570-67e726c60a81-cert\") pod \"kserve-controller-manager-66f4b8957f-zccxh\" (UID: \"fd7cfb42-7e4f-4388-9570-67e726c60a81\") " pod="kserve/kserve-controller-manager-66f4b8957f-zccxh"
Sep 17 20:58:00.960721 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:00.960699    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd7cfb42-7e4f-4388-9570-67e726c60a81-cert\") pod \"kserve-controller-manager-66f4b8957f-zccxh\" (UID: \"fd7cfb42-7e4f-4388-9570-67e726c60a81\") " pod="kserve/kserve-controller-manager-66f4b8957f-zccxh"
Sep 17 20:58:00.988605 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:00.988559    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:58:01.090248 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:01.090214    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:58:01.189939 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:01.189911    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:58:01.216194 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:01.216115    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-66f4b8957f-zccxh"
Sep 17 20:58:01.490664 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:01.490569    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-66f4b8957f-zccxh"]
Sep 17 20:58:01.493539 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:58:01.493512    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd7cfb42_7e4f_4388_9570_67e726c60a81.slice/crio-845a50cbd4e2f8be0907b51953a71c7f5b866b67cd7875ea1c340ac15c397465 WatchSource:0}: Error finding container 845a50cbd4e2f8be0907b51953a71c7f5b866b67cd7875ea1c340ac15c397465: Status 404 returned error can't find the container with id 845a50cbd4e2f8be0907b51953a71c7f5b866b67cd7875ea1c340ac15c397465
Sep 17 20:58:01.495106 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:01.495087    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 20:58:02.211280 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:02.211242    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-66f4b8957f-zccxh" event={"ID":"fd7cfb42-7e4f-4388-9570-67e726c60a81","Type":"ContainerStarted","Data":"845a50cbd4e2f8be0907b51953a71c7f5b866b67cd7875ea1c340ac15c397465"}
Sep 17 20:58:04.221656 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:04.221621    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-66f4b8957f-zccxh" event={"ID":"fd7cfb42-7e4f-4388-9570-67e726c60a81","Type":"ContainerStarted","Data":"03be52ac552bf7a273017c0ce4fa72dd079ecbeeeb1eb022ad2e213b05e78aa2"}
Sep 17 20:58:04.222062 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:04.221748    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-66f4b8957f-zccxh"
Sep 17 20:58:04.238375 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:04.238324    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-66f4b8957f-zccxh" podStartSLOduration=1.696757405 podStartE2EDuration="4.238309687s" totalImagesPullingTime="2.541552282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:58:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:58:04.237274194 +0000 UTC m=+548.582876036" watchObservedRunningTime="2026-09-17 20:58:04.238309687 +0000 UTC m=+548.583911511"
Sep 17 20:58:35.231236 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:35.231197    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-66f4b8957f-zccxh"
Sep 17 20:58:45.156436 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:45.156397    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-66f4b8957f-zccxh"]
Sep 17 20:58:45.156900 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:45.156597    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve/kserve-controller-manager-66f4b8957f-zccxh" podUID="fd7cfb42-7e4f-4388-9570-67e726c60a81" containerName="manager" containerID="cri-o://03be52ac552bf7a273017c0ce4fa72dd079ecbeeeb1eb022ad2e213b05e78aa2" gracePeriod=10
Sep 17 20:58:45.196743 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:45.196708    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-66f4b8957f-jgxts"]
Sep 17 20:58:45.199956 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:45.199935    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-66f4b8957f-jgxts"
Sep 17 20:58:45.213330 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:45.213301    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-66f4b8957f-jgxts"]
Sep 17 20:58:45.226896 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:45.226860    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve/kserve-controller-manager-66f4b8957f-zccxh" podUID="fd7cfb42-7e4f-4388-9570-67e726c60a81" containerName="manager" probeResult="failure" output="Get \"http://10.132.0.22:8081/readyz\": dial tcp 10.132.0.22:8081: connect: connection refused"
Sep 17 20:58:45.315434 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:45.315395    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1a4cfae-1c90-4053-91a1-035f03dcdae0-cert\") pod \"kserve-controller-manager-66f4b8957f-jgxts\" (UID: \"a1a4cfae-1c90-4053-91a1-035f03dcdae0\") " pod="kserve/kserve-controller-manager-66f4b8957f-jgxts"
Sep 17 20:58:45.315583 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:45.315445    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pg7tb\" (UniqueName: \"kubernetes.io/projected/a1a4cfae-1c90-4053-91a1-035f03dcdae0-kube-api-access-pg7tb\") pod \"kserve-controller-manager-66f4b8957f-jgxts\" (UID: \"a1a4cfae-1c90-4053-91a1-035f03dcdae0\") " pod="kserve/kserve-controller-manager-66f4b8957f-jgxts"
Sep 17 20:58:45.381446 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:45.381415    2567 generic.go:356] "Generic (PLEG): container finished" podID="fd7cfb42-7e4f-4388-9570-67e726c60a81" containerID="03be52ac552bf7a273017c0ce4fa72dd079ecbeeeb1eb022ad2e213b05e78aa2" exitCode=0
Sep 17 20:58:45.381556 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:45.381491    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-66f4b8957f-zccxh" event={"ID":"fd7cfb42-7e4f-4388-9570-67e726c60a81","Type":"ContainerDied","Data":"03be52ac552bf7a273017c0ce4fa72dd079ecbeeeb1eb022ad2e213b05e78aa2"}
Sep 17 20:58:45.381612 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:45.381553    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-66f4b8957f-zccxh" event={"ID":"fd7cfb42-7e4f-4388-9570-67e726c60a81","Type":"ContainerDied","Data":"845a50cbd4e2f8be0907b51953a71c7f5b866b67cd7875ea1c340ac15c397465"}
Sep 17 20:58:45.381612 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:45.381574    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="845a50cbd4e2f8be0907b51953a71c7f5b866b67cd7875ea1c340ac15c397465"
Sep 17 20:58:45.393228 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:45.393208    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-66f4b8957f-zccxh"
Sep 17 20:58:45.416369 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:45.416299    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p6l64\" (UniqueName: \"kubernetes.io/projected/fd7cfb42-7e4f-4388-9570-67e726c60a81-kube-api-access-p6l64\") pod \"fd7cfb42-7e4f-4388-9570-67e726c60a81\" (UID: \"fd7cfb42-7e4f-4388-9570-67e726c60a81\") "
Sep 17 20:58:45.416495 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:45.416371    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd7cfb42-7e4f-4388-9570-67e726c60a81-cert\") pod \"fd7cfb42-7e4f-4388-9570-67e726c60a81\" (UID: \"fd7cfb42-7e4f-4388-9570-67e726c60a81\") "
Sep 17 20:58:45.416598 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:45.416577    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1a4cfae-1c90-4053-91a1-035f03dcdae0-cert\") pod \"kserve-controller-manager-66f4b8957f-jgxts\" (UID: \"a1a4cfae-1c90-4053-91a1-035f03dcdae0\") " pod="kserve/kserve-controller-manager-66f4b8957f-jgxts"
Sep 17 20:58:45.416672 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:45.416648    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pg7tb\" (UniqueName: \"kubernetes.io/projected/a1a4cfae-1c90-4053-91a1-035f03dcdae0-kube-api-access-pg7tb\") pod \"kserve-controller-manager-66f4b8957f-jgxts\" (UID: \"a1a4cfae-1c90-4053-91a1-035f03dcdae0\") " pod="kserve/kserve-controller-manager-66f4b8957f-jgxts"
Sep 17 20:58:45.419278 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:45.419254    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1a4cfae-1c90-4053-91a1-035f03dcdae0-cert\") pod \"kserve-controller-manager-66f4b8957f-jgxts\" (UID: \"a1a4cfae-1c90-4053-91a1-035f03dcdae0\") " pod="kserve/kserve-controller-manager-66f4b8957f-jgxts"
Sep 17 20:58:45.419592 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:45.419559    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd7cfb42-7e4f-4388-9570-67e726c60a81-cert" (OuterVolumeSpecName: "cert") pod "fd7cfb42-7e4f-4388-9570-67e726c60a81" (UID: "fd7cfb42-7e4f-4388-9570-67e726c60a81"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 20:58:45.419702 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:45.419596    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd7cfb42-7e4f-4388-9570-67e726c60a81-kube-api-access-p6l64" (OuterVolumeSpecName: "kube-api-access-p6l64") pod "fd7cfb42-7e4f-4388-9570-67e726c60a81" (UID: "fd7cfb42-7e4f-4388-9570-67e726c60a81"). InnerVolumeSpecName "kube-api-access-p6l64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 20:58:45.427231 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:45.427205    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pg7tb\" (UniqueName: \"kubernetes.io/projected/a1a4cfae-1c90-4053-91a1-035f03dcdae0-kube-api-access-pg7tb\") pod \"kserve-controller-manager-66f4b8957f-jgxts\" (UID: \"a1a4cfae-1c90-4053-91a1-035f03dcdae0\") " pod="kserve/kserve-controller-manager-66f4b8957f-jgxts"
Sep 17 20:58:45.517506 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:45.517470    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6l64\" (UniqueName: \"kubernetes.io/projected/fd7cfb42-7e4f-4388-9570-67e726c60a81-kube-api-access-p6l64\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:58:45.517506 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:45.517496    2567 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd7cfb42-7e4f-4388-9570-67e726c60a81-cert\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 20:58:45.551646 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:45.551603    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-66f4b8957f-jgxts"
Sep 17 20:58:45.824364 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:45.824247    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-66f4b8957f-jgxts"]
Sep 17 20:58:45.826602 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:58:45.826573    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1a4cfae_1c90_4053_91a1_035f03dcdae0.slice/crio-06cc04eeb56e293f44a031f4f9b865aa20ad7b062c7130da2b05abc007331477 WatchSource:0}: Error finding container 06cc04eeb56e293f44a031f4f9b865aa20ad7b062c7130da2b05abc007331477: Status 404 returned error can't find the container with id 06cc04eeb56e293f44a031f4f9b865aa20ad7b062c7130da2b05abc007331477
Sep 17 20:58:46.387291 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:46.387261    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-66f4b8957f-zccxh"
Sep 17 20:58:46.387742 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:46.387261    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-66f4b8957f-jgxts" event={"ID":"a1a4cfae-1c90-4053-91a1-035f03dcdae0","Type":"ContainerStarted","Data":"1545e8c97e06c6598ca16913786d8730351bd09dac7c07665c26e29d9218d5a6"}
Sep 17 20:58:46.387742 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:46.387448    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-66f4b8957f-jgxts" event={"ID":"a1a4cfae-1c90-4053-91a1-035f03dcdae0","Type":"ContainerStarted","Data":"06cc04eeb56e293f44a031f4f9b865aa20ad7b062c7130da2b05abc007331477"}
Sep 17 20:58:46.387742 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:46.387580    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-66f4b8957f-jgxts"
Sep 17 20:58:46.409385 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:46.409331    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-66f4b8957f-jgxts" podStartSLOduration=1.042131561 podStartE2EDuration="1.409317189s" totalImagesPullingTime="367.185628ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:58:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:58:46.408969884 +0000 UTC m=+590.754571727" watchObservedRunningTime="2026-09-17 20:58:46.409317189 +0000 UTC m=+590.754919014"
Sep 17 20:58:46.423710 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:46.423683    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-66f4b8957f-zccxh"]
Sep 17 20:58:46.425950 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:46.425929    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-66f4b8957f-zccxh"]
Sep 17 20:58:48.169608 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:58:48.169579    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd7cfb42-7e4f-4388-9570-67e726c60a81" path="/var/lib/kubelet/pods/fd7cfb42-7e4f-4388-9570-67e726c60a81/volumes"
Sep 17 20:59:17.396905 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:17.396870    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-66f4b8957f-jgxts"
Sep 17 20:59:18.636695 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:18.636587    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:59:18.668701 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:18.668670    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-7b954ffc95-wwj5c"]
Sep 17 20:59:18.669018 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:18.669002    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd7cfb42-7e4f-4388-9570-67e726c60a81" containerName="manager"
Sep 17 20:59:18.669063 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:18.669019    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd7cfb42-7e4f-4388-9570-67e726c60a81" containerName="manager"
Sep 17 20:59:18.669100 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:18.669091    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd7cfb42-7e4f-4388-9570-67e726c60a81" containerName="manager"
Sep 17 20:59:18.671837 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:18.671819    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-wwj5c"
Sep 17 20:59:18.674458 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:18.674439    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\""
Sep 17 20:59:18.674560 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:18.674444    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-vbf7g\""
Sep 17 20:59:18.681793 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:18.681771    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-wwj5c"]
Sep 17 20:59:18.710333 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:18.710299    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:59:18.772607 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:18.772558    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:59:18.783563 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:18.783536    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ace0b21-86b2-408d-8d11-9e708a1f24af-tls-certs\") pod \"model-serving-api-7b954ffc95-wwj5c\" (UID: \"7ace0b21-86b2-408d-8d11-9e708a1f24af\") " pod="kserve/model-serving-api-7b954ffc95-wwj5c"
Sep 17 20:59:18.783689 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:18.783564    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r6s4\" (UniqueName: \"kubernetes.io/projected/7ace0b21-86b2-408d-8d11-9e708a1f24af-kube-api-access-7r6s4\") pod \"model-serving-api-7b954ffc95-wwj5c\" (UID: \"7ace0b21-86b2-408d-8d11-9e708a1f24af\") " pod="kserve/model-serving-api-7b954ffc95-wwj5c"
Sep 17 20:59:18.884711 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:18.884677    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ace0b21-86b2-408d-8d11-9e708a1f24af-tls-certs\") pod \"model-serving-api-7b954ffc95-wwj5c\" (UID: \"7ace0b21-86b2-408d-8d11-9e708a1f24af\") " pod="kserve/model-serving-api-7b954ffc95-wwj5c"
Sep 17 20:59:18.884711 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:18.884713    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r6s4\" (UniqueName: \"kubernetes.io/projected/7ace0b21-86b2-408d-8d11-9e708a1f24af-kube-api-access-7r6s4\") pod \"model-serving-api-7b954ffc95-wwj5c\" (UID: \"7ace0b21-86b2-408d-8d11-9e708a1f24af\") " pod="kserve/model-serving-api-7b954ffc95-wwj5c"
Sep 17 20:59:18.884976 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:59:18.884838    2567 secret.go:189] Couldn't get secret kserve/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 17 20:59:18.884976 ip-10-0-132-201 kubenswrapper[2567]: E0917 20:59:18.884918    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ace0b21-86b2-408d-8d11-9e708a1f24af-tls-certs podName:7ace0b21-86b2-408d-8d11-9e708a1f24af nodeName:}" failed. No retries permitted until 2026-09-17 20:59:19.384901429 +0000 UTC m=+623.730503236 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/7ace0b21-86b2-408d-8d11-9e708a1f24af-tls-certs") pod "model-serving-api-7b954ffc95-wwj5c" (UID: "7ace0b21-86b2-408d-8d11-9e708a1f24af") : secret "model-serving-api-tls" not found
Sep 17 20:59:18.896005 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:18.895947    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r6s4\" (UniqueName: \"kubernetes.io/projected/7ace0b21-86b2-408d-8d11-9e708a1f24af-kube-api-access-7r6s4\") pod \"model-serving-api-7b954ffc95-wwj5c\" (UID: \"7ace0b21-86b2-408d-8d11-9e708a1f24af\") " pod="kserve/model-serving-api-7b954ffc95-wwj5c"
Sep 17 20:59:18.952710 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:18.952661    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:59:19.052449 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:19.052419    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:59:19.389014 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:19.388975    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ace0b21-86b2-408d-8d11-9e708a1f24af-tls-certs\") pod \"model-serving-api-7b954ffc95-wwj5c\" (UID: \"7ace0b21-86b2-408d-8d11-9e708a1f24af\") " pod="kserve/model-serving-api-7b954ffc95-wwj5c"
Sep 17 20:59:19.391404 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:19.391374    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ace0b21-86b2-408d-8d11-9e708a1f24af-tls-certs\") pod \"model-serving-api-7b954ffc95-wwj5c\" (UID: \"7ace0b21-86b2-408d-8d11-9e708a1f24af\") " pod="kserve/model-serving-api-7b954ffc95-wwj5c"
Sep 17 20:59:19.582735 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:19.582697    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-wwj5c"
Sep 17 20:59:19.775358 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:19.775328    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:59:19.854825 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:19.854801    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-wwj5c"]
Sep 17 20:59:19.857392 ip-10-0-132-201 kubenswrapper[2567]: W0917 20:59:19.857367    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ace0b21_86b2_408d_8d11_9e708a1f24af.slice/crio-38395fbc4b3b72adace96b13b4834f47c9101eeefa2dead522728d5833aa8af8 WatchSource:0}: Error finding container 38395fbc4b3b72adace96b13b4834f47c9101eeefa2dead522728d5833aa8af8: Status 404 returned error can't find the container with id 38395fbc4b3b72adace96b13b4834f47c9101eeefa2dead522728d5833aa8af8
Sep 17 20:59:19.857846 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:19.857815    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 20:59:20.519394 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:20.519353    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-wwj5c" event={"ID":"7ace0b21-86b2-408d-8d11-9e708a1f24af","Type":"ContainerStarted","Data":"38395fbc4b3b72adace96b13b4834f47c9101eeefa2dead522728d5833aa8af8"}
Sep 17 20:59:22.530377 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:22.530337    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-wwj5c" event={"ID":"7ace0b21-86b2-408d-8d11-9e708a1f24af","Type":"ContainerStarted","Data":"c9e2a626e176bb8315f293dc815a00858c61a4476e1d0e748112553107d78776"}
Sep 17 20:59:22.530847 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:22.530445    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-7b954ffc95-wwj5c"
Sep 17 20:59:22.549473 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:22.549428    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-7b954ffc95-wwj5c" podStartSLOduration=2.261901802 podStartE2EDuration="4.549413376s" totalImagesPullingTime="2.287511574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:59:22.549301409 +0000 UTC m=+626.894903247" watchObservedRunningTime="2026-09-17 20:59:22.549413376 +0000 UTC m=+626.895015197"
Sep 17 20:59:33.538822 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:33.538785    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-7b954ffc95-wwj5c"
Sep 17 20:59:44.668040 ip-10-0-132-201 kubenswrapper[2567]: I0917 20:59:44.668005    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:00:26.940401 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:00:26.940352    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:00:54.442084 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:00:54.441994    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:00:54.542628 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:00:54.542594    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:01:07.271373 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.271338    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:01:07.376807 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.376769    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:01:07.479828 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.479789    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:01:07.561398 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.561324    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp"]
Sep 17 21:01:07.564259 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.564223    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:01:07.564793 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.564776    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp"
Sep 17 21:01:07.567740 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.567716    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Sep 17 21:01:07.567871 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.567722    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Sep 17 21:01:07.567871 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.567776    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-rollback-kube-rbac-proxy-sar-config\""
Sep 17 21:01:07.568094 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.568078    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 17 21:01:07.569322 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.569302    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-rollback-predictor-serving-cert\""
Sep 17 21:01:07.569322 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.569314    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 17 21:01:07.569474 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.569326    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-5dj6n\""
Sep 17 21:01:07.580470 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.580450    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp"]
Sep 17 21:01:07.638918 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.638885    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:01:07.671777 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.671731    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/97bc6251-5ff3-4395-bf0e-606fdb336f51-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-97977b6b5-cp7gp\" (UID: \"97bc6251-5ff3-4395-bf0e-606fdb336f51\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp"
Sep 17 21:01:07.671777 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.671778    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnzt7\" (UniqueName: \"kubernetes.io/projected/97bc6251-5ff3-4395-bf0e-606fdb336f51-kube-api-access-dnzt7\") pod \"isvc-canary-rollback-predictor-97977b6b5-cp7gp\" (UID: \"97bc6251-5ff3-4395-bf0e-606fdb336f51\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp"
Sep 17 21:01:07.672012 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.671810    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/97bc6251-5ff3-4395-bf0e-606fdb336f51-cabundle-cert\") pod \"isvc-canary-rollback-predictor-97977b6b5-cp7gp\" (UID: \"97bc6251-5ff3-4395-bf0e-606fdb336f51\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp"
Sep 17 21:01:07.672012 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.671880    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/97bc6251-5ff3-4395-bf0e-606fdb336f51-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-97977b6b5-cp7gp\" (UID: \"97bc6251-5ff3-4395-bf0e-606fdb336f51\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp"
Sep 17 21:01:07.672012 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.671950    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/97bc6251-5ff3-4395-bf0e-606fdb336f51-proxy-tls\") pod \"isvc-canary-rollback-predictor-97977b6b5-cp7gp\" (UID: \"97bc6251-5ff3-4395-bf0e-606fdb336f51\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp"
Sep 17 21:01:07.746447 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.746411    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:01:07.772414 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.772380    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnzt7\" (UniqueName: \"kubernetes.io/projected/97bc6251-5ff3-4395-bf0e-606fdb336f51-kube-api-access-dnzt7\") pod \"isvc-canary-rollback-predictor-97977b6b5-cp7gp\" (UID: \"97bc6251-5ff3-4395-bf0e-606fdb336f51\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp"
Sep 17 21:01:07.772564 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.772419    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/97bc6251-5ff3-4395-bf0e-606fdb336f51-cabundle-cert\") pod \"isvc-canary-rollback-predictor-97977b6b5-cp7gp\" (UID: \"97bc6251-5ff3-4395-bf0e-606fdb336f51\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp"
Sep 17 21:01:07.772564 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.772444    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/97bc6251-5ff3-4395-bf0e-606fdb336f51-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-97977b6b5-cp7gp\" (UID: \"97bc6251-5ff3-4395-bf0e-606fdb336f51\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp"
Sep 17 21:01:07.772564 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.772478    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/97bc6251-5ff3-4395-bf0e-606fdb336f51-proxy-tls\") pod \"isvc-canary-rollback-predictor-97977b6b5-cp7gp\" (UID: \"97bc6251-5ff3-4395-bf0e-606fdb336f51\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp"
Sep 17 21:01:07.772564 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.772516    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/97bc6251-5ff3-4395-bf0e-606fdb336f51-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-97977b6b5-cp7gp\" (UID: \"97bc6251-5ff3-4395-bf0e-606fdb336f51\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp"
Sep 17 21:01:07.773007 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.772981    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/97bc6251-5ff3-4395-bf0e-606fdb336f51-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-97977b6b5-cp7gp\" (UID: \"97bc6251-5ff3-4395-bf0e-606fdb336f51\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp"
Sep 17 21:01:07.773109 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.773089    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/97bc6251-5ff3-4395-bf0e-606fdb336f51-cabundle-cert\") pod \"isvc-canary-rollback-predictor-97977b6b5-cp7gp\" (UID: \"97bc6251-5ff3-4395-bf0e-606fdb336f51\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp"
Sep 17 21:01:07.773233 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.773216    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/97bc6251-5ff3-4395-bf0e-606fdb336f51-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-97977b6b5-cp7gp\" (UID: \"97bc6251-5ff3-4395-bf0e-606fdb336f51\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp"
Sep 17 21:01:07.775087 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.775059    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/97bc6251-5ff3-4395-bf0e-606fdb336f51-proxy-tls\") pod \"isvc-canary-rollback-predictor-97977b6b5-cp7gp\" (UID: \"97bc6251-5ff3-4395-bf0e-606fdb336f51\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp"
Sep 17 21:01:07.782463 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.782444    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnzt7\" (UniqueName: \"kubernetes.io/projected/97bc6251-5ff3-4395-bf0e-606fdb336f51-kube-api-access-dnzt7\") pod \"isvc-canary-rollback-predictor-97977b6b5-cp7gp\" (UID: \"97bc6251-5ff3-4395-bf0e-606fdb336f51\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp"
Sep 17 21:01:07.875681 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:07.875582    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp"
Sep 17 21:01:08.158044 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:08.158018    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp"]
Sep 17 21:01:08.159778 ip-10-0-132-201 kubenswrapper[2567]: W0917 21:01:08.159734    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97bc6251_5ff3_4395_bf0e_606fdb336f51.slice/crio-92741fc91c6d0ec02bcd63ff5d8bbac8d2846ab91c88caf2d8578f3946fc253f WatchSource:0}: Error finding container 92741fc91c6d0ec02bcd63ff5d8bbac8d2846ab91c88caf2d8578f3946fc253f: Status 404 returned error can't find the container with id 92741fc91c6d0ec02bcd63ff5d8bbac8d2846ab91c88caf2d8578f3946fc253f
Sep 17 21:01:08.943557 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:08.943524    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp" event={"ID":"97bc6251-5ff3-4395-bf0e-606fdb336f51","Type":"ContainerStarted","Data":"92741fc91c6d0ec02bcd63ff5d8bbac8d2846ab91c88caf2d8578f3946fc253f"}
Sep 17 21:01:12.962970 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:12.962931    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp" event={"ID":"97bc6251-5ff3-4395-bf0e-606fdb336f51","Type":"ContainerStarted","Data":"f458f1f2866ca44e816ab156a709bdd79b4ce4178d3095d2f8d010d7075412e3"}
Sep 17 21:01:16.980327 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:16.980295    2567 generic.go:356] "Generic (PLEG): container finished" podID="97bc6251-5ff3-4395-bf0e-606fdb336f51" containerID="f458f1f2866ca44e816ab156a709bdd79b4ce4178d3095d2f8d010d7075412e3" exitCode=0
Sep 17 21:01:16.980710 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:16.980333    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp" event={"ID":"97bc6251-5ff3-4395-bf0e-606fdb336f51","Type":"ContainerDied","Data":"f458f1f2866ca44e816ab156a709bdd79b4ce4178d3095d2f8d010d7075412e3"}
Sep 17 21:01:31.058345 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:31.058259    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp" event={"ID":"97bc6251-5ff3-4395-bf0e-606fdb336f51","Type":"ContainerStarted","Data":"9fac5e0ec96d46b5c92f71ebcbe8589e2279ca930a48fafa48a645a072d8e4e5"}
Sep 17 21:01:34.075177 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:34.075140    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp" event={"ID":"97bc6251-5ff3-4395-bf0e-606fdb336f51","Type":"ContainerStarted","Data":"4c225c5dfaaef669137096787bd07da804ea8ecf5f5e8f6ff12d210243000c6c"}
Sep 17 21:01:34.075562 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:34.075240    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp"
Sep 17 21:01:34.095946 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:34.095893    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp" podStartSLOduration=2.827563975 podStartE2EDuration="27.095877655s" totalImagesPullingTime="20.383232149s" totalInitContainerRuntime="3.885081531s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:01:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:01:34.094212853 +0000 UTC m=+758.439814697" watchObservedRunningTime="2026-09-17 21:01:34.095877655 +0000 UTC m=+758.441479480"
Sep 17 21:01:35.079149 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:35.079113    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp"
Sep 17 21:01:35.080334 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:35.080307    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp" podUID="97bc6251-5ff3-4395-bf0e-606fdb336f51" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused"
Sep 17 21:01:36.083300 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:36.083260    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp" podUID="97bc6251-5ff3-4395-bf0e-606fdb336f51" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused"
Sep 17 21:01:37.945496 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:37.945458    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:01:39.737494 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:39.737461    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:01:41.087955 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:41.087918    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp"
Sep 17 21:01:41.088566 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:41.088538    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp" podUID="97bc6251-5ff3-4395-bf0e-606fdb336f51" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused"
Sep 17 21:01:51.089351 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:01:51.089307    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp" podUID="97bc6251-5ff3-4395-bf0e-606fdb336f51" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused"
Sep 17 21:02:01.088557 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:02:01.088517    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp" podUID="97bc6251-5ff3-4395-bf0e-606fdb336f51" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused"
Sep 17 21:02:11.089325 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:02:11.089278    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp" podUID="97bc6251-5ff3-4395-bf0e-606fdb336f51" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused"
Sep 17 21:02:21.089131 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:02:21.089089    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp" podUID="97bc6251-5ff3-4395-bf0e-606fdb336f51" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused"
Sep 17 21:02:21.647346 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:02:21.647312    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:02:31.088775 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:02:31.088714    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp" podUID="97bc6251-5ff3-4395-bf0e-606fdb336f51" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused"
Sep 17 21:02:41.089097 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:02:41.089052    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp" podUID="97bc6251-5ff3-4395-bf0e-606fdb336f51" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused"
Sep 17 21:02:51.089916 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:02:51.089885    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp"
Sep 17 21:03:02.225163 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:02.225129    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:03:12.774402 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:12.774370    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:03:12.871282 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:12.871246    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:03:12.888189 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:12.888162    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h"]
Sep 17 21:03:12.892057 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:12.892041    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h"
Sep 17 21:03:12.894703 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:12.894684    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-raw-669c3-predictor-serving-cert\""
Sep 17 21:03:12.894894 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:12.894876    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-raw-669c3-kube-rbac-proxy-sar-config\""
Sep 17 21:03:12.904852 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:12.904828    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h"]
Sep 17 21:03:12.928646 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:12.928622    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e5f88102-8b3a-48d6-9f09-96aae9f59be9-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h\" (UID: \"e5f88102-8b3a-48d6-9f09-96aae9f59be9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h"
Sep 17 21:03:12.928792 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:12.928661    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e5f88102-8b3a-48d6-9f09-96aae9f59be9-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h\" (UID: \"e5f88102-8b3a-48d6-9f09-96aae9f59be9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h"
Sep 17 21:03:12.928792 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:12.928688    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6hkm\" (UniqueName: \"kubernetes.io/projected/e5f88102-8b3a-48d6-9f09-96aae9f59be9-kube-api-access-t6hkm\") pod \"isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h\" (UID: \"e5f88102-8b3a-48d6-9f09-96aae9f59be9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h"
Sep 17 21:03:12.928921 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:12.928780    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e5f88102-8b3a-48d6-9f09-96aae9f59be9-proxy-tls\") pod \"isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h\" (UID: \"e5f88102-8b3a-48d6-9f09-96aae9f59be9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h"
Sep 17 21:03:12.928921 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:12.928886    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e5f88102-8b3a-48d6-9f09-96aae9f59be9-cabundle-cert\") pod \"isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h\" (UID: \"e5f88102-8b3a-48d6-9f09-96aae9f59be9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h"
Sep 17 21:03:13.010500 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.010464    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:03:13.029581 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.029509    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns"]
Sep 17 21:03:13.029774 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.029741    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6hkm\" (UniqueName: \"kubernetes.io/projected/e5f88102-8b3a-48d6-9f09-96aae9f59be9-kube-api-access-t6hkm\") pod \"isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h\" (UID: \"e5f88102-8b3a-48d6-9f09-96aae9f59be9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h"
Sep 17 21:03:13.029844 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.029802    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e5f88102-8b3a-48d6-9f09-96aae9f59be9-proxy-tls\") pod \"isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h\" (UID: \"e5f88102-8b3a-48d6-9f09-96aae9f59be9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h"
Sep 17 21:03:13.029901 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.029869    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e5f88102-8b3a-48d6-9f09-96aae9f59be9-cabundle-cert\") pod \"isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h\" (UID: \"e5f88102-8b3a-48d6-9f09-96aae9f59be9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h"
Sep 17 21:03:13.029954 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.029905    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e5f88102-8b3a-48d6-9f09-96aae9f59be9-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h\" (UID: \"e5f88102-8b3a-48d6-9f09-96aae9f59be9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h"
Sep 17 21:03:13.030168 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.030132    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e5f88102-8b3a-48d6-9f09-96aae9f59be9-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h\" (UID: \"e5f88102-8b3a-48d6-9f09-96aae9f59be9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h"
Sep 17 21:03:13.030539 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.030510    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e5f88102-8b3a-48d6-9f09-96aae9f59be9-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h\" (UID: \"e5f88102-8b3a-48d6-9f09-96aae9f59be9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h"
Sep 17 21:03:13.030724 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.030670    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e5f88102-8b3a-48d6-9f09-96aae9f59be9-cabundle-cert\") pod \"isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h\" (UID: \"e5f88102-8b3a-48d6-9f09-96aae9f59be9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h"
Sep 17 21:03:13.030822 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.030745    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e5f88102-8b3a-48d6-9f09-96aae9f59be9-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h\" (UID: \"e5f88102-8b3a-48d6-9f09-96aae9f59be9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h"
Sep 17 21:03:13.032480 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.032461    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e5f88102-8b3a-48d6-9f09-96aae9f59be9-proxy-tls\") pod \"isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h\" (UID: \"e5f88102-8b3a-48d6-9f09-96aae9f59be9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h"
Sep 17 21:03:13.033712 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.033699    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns"
Sep 17 21:03:13.036317 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.036299    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-raw-669c3-predictor-serving-cert\""
Sep 17 21:03:13.036410 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.036336    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-raw-669c3-kube-rbac-proxy-sar-config\""
Sep 17 21:03:13.040624 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.040601    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6hkm\" (UniqueName: \"kubernetes.io/projected/e5f88102-8b3a-48d6-9f09-96aae9f59be9-kube-api-access-t6hkm\") pod \"isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h\" (UID: \"e5f88102-8b3a-48d6-9f09-96aae9f59be9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h"
Sep 17 21:03:13.051044 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.051018    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns"]
Sep 17 21:03:13.112870 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.112833    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:03:13.130609 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.130581    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a7079774-6102-4c26-b948-55bdaf0bd825-proxy-tls\") pod \"isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns\" (UID: \"a7079774-6102-4c26-b948-55bdaf0bd825\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns"
Sep 17 21:03:13.130747 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.130634    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlpjs\" (UniqueName: \"kubernetes.io/projected/a7079774-6102-4c26-b948-55bdaf0bd825-kube-api-access-tlpjs\") pod \"isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns\" (UID: \"a7079774-6102-4c26-b948-55bdaf0bd825\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns"
Sep 17 21:03:13.130747 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.130693    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a7079774-6102-4c26-b948-55bdaf0bd825-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns\" (UID: \"a7079774-6102-4c26-b948-55bdaf0bd825\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns"
Sep 17 21:03:13.130747 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.130738    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a7079774-6102-4c26-b948-55bdaf0bd825-cabundle-cert\") pod \"isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns\" (UID: \"a7079774-6102-4c26-b948-55bdaf0bd825\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns"
Sep 17 21:03:13.130929 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.130802    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a7079774-6102-4c26-b948-55bdaf0bd825-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns\" (UID: \"a7079774-6102-4c26-b948-55bdaf0bd825\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns"
Sep 17 21:03:13.202230 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.202194    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h"
Sep 17 21:03:13.212596 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.212416    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:03:13.231444 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.231416    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a7079774-6102-4c26-b948-55bdaf0bd825-cabundle-cert\") pod \"isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns\" (UID: \"a7079774-6102-4c26-b948-55bdaf0bd825\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns"
Sep 17 21:03:13.231586 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.231462    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a7079774-6102-4c26-b948-55bdaf0bd825-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns\" (UID: \"a7079774-6102-4c26-b948-55bdaf0bd825\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns"
Sep 17 21:03:13.231586 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.231506    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a7079774-6102-4c26-b948-55bdaf0bd825-proxy-tls\") pod \"isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns\" (UID: \"a7079774-6102-4c26-b948-55bdaf0bd825\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns"
Sep 17 21:03:13.231586 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.231549    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlpjs\" (UniqueName: \"kubernetes.io/projected/a7079774-6102-4c26-b948-55bdaf0bd825-kube-api-access-tlpjs\") pod \"isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns\" (UID: \"a7079774-6102-4c26-b948-55bdaf0bd825\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns"
Sep 17 21:03:13.231867 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.231818    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a7079774-6102-4c26-b948-55bdaf0bd825-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns\" (UID: \"a7079774-6102-4c26-b948-55bdaf0bd825\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns"
Sep 17 21:03:13.232170 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.232146    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a7079774-6102-4c26-b948-55bdaf0bd825-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns\" (UID: \"a7079774-6102-4c26-b948-55bdaf0bd825\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns"
Sep 17 21:03:13.232278 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.232206    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a7079774-6102-4c26-b948-55bdaf0bd825-cabundle-cert\") pod \"isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns\" (UID: \"a7079774-6102-4c26-b948-55bdaf0bd825\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns"
Sep 17 21:03:13.232524 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.232497    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a7079774-6102-4c26-b948-55bdaf0bd825-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns\" (UID: \"a7079774-6102-4c26-b948-55bdaf0bd825\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns"
Sep 17 21:03:13.233905 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.233884    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a7079774-6102-4c26-b948-55bdaf0bd825-proxy-tls\") pod \"isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns\" (UID: \"a7079774-6102-4c26-b948-55bdaf0bd825\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns"
Sep 17 21:03:13.241308 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.241282    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlpjs\" (UniqueName: \"kubernetes.io/projected/a7079774-6102-4c26-b948-55bdaf0bd825-kube-api-access-tlpjs\") pod \"isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns\" (UID: \"a7079774-6102-4c26-b948-55bdaf0bd825\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns"
Sep 17 21:03:13.355971 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.355894    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns"
Sep 17 21:03:13.476925 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.476842    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h"]
Sep 17 21:03:13.479649 ip-10-0-132-201 kubenswrapper[2567]: W0917 21:03:13.479615    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5f88102_8b3a_48d6_9f09_96aae9f59be9.slice/crio-50a8c9a578ac9c56daea6335af403f6c84b57420eddbc7080e1efb1944ff8542 WatchSource:0}: Error finding container 50a8c9a578ac9c56daea6335af403f6c84b57420eddbc7080e1efb1944ff8542: Status 404 returned error can't find the container with id 50a8c9a578ac9c56daea6335af403f6c84b57420eddbc7080e1efb1944ff8542
Sep 17 21:03:13.629746 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:13.629720    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns"]
Sep 17 21:03:13.632144 ip-10-0-132-201 kubenswrapper[2567]: W0917 21:03:13.632121    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7079774_6102_4c26_b948_55bdaf0bd825.slice/crio-12e5e51b5819a91057717c63aef2a7cc38ba6fa3187a812c404c69100e651628 WatchSource:0}: Error finding container 12e5e51b5819a91057717c63aef2a7cc38ba6fa3187a812c404c69100e651628: Status 404 returned error can't find the container with id 12e5e51b5819a91057717c63aef2a7cc38ba6fa3187a812c404c69100e651628
Sep 17 21:03:14.467970 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:14.467927    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns" event={"ID":"a7079774-6102-4c26-b948-55bdaf0bd825","Type":"ContainerStarted","Data":"1107586fab4aa59c3a77fc0591771b931777afee9945a9484c1468246d3ef0a7"}
Sep 17 21:03:14.468426 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:14.467999    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns" event={"ID":"a7079774-6102-4c26-b948-55bdaf0bd825","Type":"ContainerStarted","Data":"12e5e51b5819a91057717c63aef2a7cc38ba6fa3187a812c404c69100e651628"}
Sep 17 21:03:14.469596 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:14.469569    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h" event={"ID":"e5f88102-8b3a-48d6-9f09-96aae9f59be9","Type":"ContainerStarted","Data":"dda92f4aef5340bb644e5e8cf59ba3308a3c7947b4b869f864c4f00de2d3ea6c"}
Sep 17 21:03:14.469694 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:14.469622    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h" event={"ID":"e5f88102-8b3a-48d6-9f09-96aae9f59be9","Type":"ContainerStarted","Data":"50a8c9a578ac9c56daea6335af403f6c84b57420eddbc7080e1efb1944ff8542"}
Sep 17 21:03:17.484732 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:17.484697    2567 generic.go:356] "Generic (PLEG): container finished" podID="a7079774-6102-4c26-b948-55bdaf0bd825" containerID="1107586fab4aa59c3a77fc0591771b931777afee9945a9484c1468246d3ef0a7" exitCode=0
Sep 17 21:03:17.485198 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:17.484783    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns" event={"ID":"a7079774-6102-4c26-b948-55bdaf0bd825","Type":"ContainerDied","Data":"1107586fab4aa59c3a77fc0591771b931777afee9945a9484c1468246d3ef0a7"}
Sep 17 21:03:17.486412 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:17.486390    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 21:03:17.486593 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:17.486575    2567 generic.go:356] "Generic (PLEG): container finished" podID="e5f88102-8b3a-48d6-9f09-96aae9f59be9" containerID="dda92f4aef5340bb644e5e8cf59ba3308a3c7947b4b869f864c4f00de2d3ea6c" exitCode=0
Sep 17 21:03:17.486661 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:17.486619    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h" event={"ID":"e5f88102-8b3a-48d6-9f09-96aae9f59be9","Type":"ContainerDied","Data":"dda92f4aef5340bb644e5e8cf59ba3308a3c7947b4b869f864c4f00de2d3ea6c"}
Sep 17 21:03:18.494548 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:18.494516    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h" event={"ID":"e5f88102-8b3a-48d6-9f09-96aae9f59be9","Type":"ContainerStarted","Data":"a0215c4c0c11c4ae7231a34ce6e413cf98a4e04e0dbbcae21dd792953393d865"}
Sep 17 21:03:18.495019 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:18.494581    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h" event={"ID":"e5f88102-8b3a-48d6-9f09-96aae9f59be9","Type":"ContainerStarted","Data":"f9b2917588a2fb4ed6d4523ff108a4532ca0d3089edc0859a08f7c482697eeb8"}
Sep 17 21:03:18.495019 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:18.494895    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h"
Sep 17 21:03:18.522989 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:18.522942    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h" podStartSLOduration=2.695688078 podStartE2EDuration="6.522926972s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.827238894s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:03:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:03:18.520449447 +0000 UTC m=+862.866051297" watchObservedRunningTime="2026-09-17 21:03:18.522926972 +0000 UTC m=+862.868528851"
Sep 17 21:03:19.500187 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:19.500151    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h"
Sep 17 21:03:19.501843 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:19.501810    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h" podUID="e5f88102-8b3a-48d6-9f09-96aae9f59be9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Sep 17 21:03:20.505205 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:20.505158    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h" podUID="e5f88102-8b3a-48d6-9f09-96aae9f59be9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Sep 17 21:03:25.509905 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:25.509870    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h"
Sep 17 21:03:25.510479 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:25.510450    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h" podUID="e5f88102-8b3a-48d6-9f09-96aae9f59be9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Sep 17 21:03:35.511118 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:35.511079    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h" podUID="e5f88102-8b3a-48d6-9f09-96aae9f59be9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Sep 17 21:03:36.596292 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:36.596253    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns" event={"ID":"a7079774-6102-4c26-b948-55bdaf0bd825","Type":"ContainerStarted","Data":"c88022be5f3a814ea5c93e18d5fef48906da7125b3c9468a4e19d66473cfa243"}
Sep 17 21:03:36.596678 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:36.596315    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns" event={"ID":"a7079774-6102-4c26-b948-55bdaf0bd825","Type":"ContainerStarted","Data":"9d72fa55fea2305eb398278d067991dd019a4e4f4aa3419914f05e400bfcc0cd"}
Sep 17 21:03:36.596678 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:36.596640    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns"
Sep 17 21:03:36.596787 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:36.596697    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns"
Sep 17 21:03:36.598068 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:36.598040    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns" podUID="a7079774-6102-4c26-b948-55bdaf0bd825" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused"
Sep 17 21:03:36.619085 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:36.619041    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns" podStartSLOduration=1.6062302499999999 podStartE2EDuration="23.619028779s" totalImagesPullingTime="18.413061213s" totalInitContainerRuntime="3.599737316s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:03:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:03:36.617983303 +0000 UTC m=+880.963585148" watchObservedRunningTime="2026-09-17 21:03:36.619028779 +0000 UTC m=+880.964630605"
Sep 17 21:03:37.601087 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:37.601045    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns" podUID="a7079774-6102-4c26-b948-55bdaf0bd825" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused"
Sep 17 21:03:42.606700 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:42.606671    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns"
Sep 17 21:03:42.607204 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:42.607174    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns" podUID="a7079774-6102-4c26-b948-55bdaf0bd825" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused"
Sep 17 21:03:45.510896 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:45.510856    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h" podUID="e5f88102-8b3a-48d6-9f09-96aae9f59be9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Sep 17 21:03:52.607728 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:52.607646    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns" podUID="a7079774-6102-4c26-b948-55bdaf0bd825" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused"
Sep 17 21:03:55.510796 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:03:55.510734    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h" podUID="e5f88102-8b3a-48d6-9f09-96aae9f59be9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Sep 17 21:04:02.607591 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:02.607548    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns" podUID="a7079774-6102-4c26-b948-55bdaf0bd825" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused"
Sep 17 21:04:05.510578 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:05.510537    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h" podUID="e5f88102-8b3a-48d6-9f09-96aae9f59be9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Sep 17 21:04:12.607643 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:12.607602    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns" podUID="a7079774-6102-4c26-b948-55bdaf0bd825" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused"
Sep 17 21:04:15.511197 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:15.511155    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h" podUID="e5f88102-8b3a-48d6-9f09-96aae9f59be9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Sep 17 21:04:22.318587 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:22.318547    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp"]
Sep 17 21:04:22.321799 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:22.318877    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp" podUID="97bc6251-5ff3-4395-bf0e-606fdb336f51" containerName="kserve-container" containerID="cri-o://9fac5e0ec96d46b5c92f71ebcbe8589e2279ca930a48fafa48a645a072d8e4e5" gracePeriod=30
Sep 17 21:04:22.321799 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:22.318992    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp" podUID="97bc6251-5ff3-4395-bf0e-606fdb336f51" containerName="kube-rbac-proxy" containerID="cri-o://4c225c5dfaaef669137096787bd07da804ea8ecf5f5e8f6ff12d210243000c6c" gracePeriod=30
Sep 17 21:04:22.341385 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:22.341353    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:04:22.608069 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:22.607978    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns" podUID="a7079774-6102-4c26-b948-55bdaf0bd825" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused"
Sep 17 21:04:22.798467 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:22.798435    2567 generic.go:356] "Generic (PLEG): container finished" podID="97bc6251-5ff3-4395-bf0e-606fdb336f51" containerID="4c225c5dfaaef669137096787bd07da804ea8ecf5f5e8f6ff12d210243000c6c" exitCode=2
Sep 17 21:04:22.798629 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:22.798510    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp" event={"ID":"97bc6251-5ff3-4395-bf0e-606fdb336f51","Type":"ContainerDied","Data":"4c225c5dfaaef669137096787bd07da804ea8ecf5f5e8f6ff12d210243000c6c"}
Sep 17 21:04:25.510973 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:25.510934    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h" podUID="e5f88102-8b3a-48d6-9f09-96aae9f59be9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Sep 17 21:04:26.084082 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:26.084042    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp" podUID="97bc6251-5ff3-4395-bf0e-606fdb336f51" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.25:8643/healthz\": dial tcp 10.132.0.25:8643: connect: connection refused"
Sep 17 21:04:27.568982 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.568956    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp"
Sep 17 21:04:27.625101 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.624998    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:04:27.695328 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.695299    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/97bc6251-5ff3-4395-bf0e-606fdb336f51-proxy-tls\") pod \"97bc6251-5ff3-4395-bf0e-606fdb336f51\" (UID: \"97bc6251-5ff3-4395-bf0e-606fdb336f51\") "
Sep 17 21:04:27.695504 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.695344    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dnzt7\" (UniqueName: \"kubernetes.io/projected/97bc6251-5ff3-4395-bf0e-606fdb336f51-kube-api-access-dnzt7\") pod \"97bc6251-5ff3-4395-bf0e-606fdb336f51\" (UID: \"97bc6251-5ff3-4395-bf0e-606fdb336f51\") "
Sep 17 21:04:27.695504 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.695378    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/97bc6251-5ff3-4395-bf0e-606fdb336f51-kserve-provision-location\") pod \"97bc6251-5ff3-4395-bf0e-606fdb336f51\" (UID: \"97bc6251-5ff3-4395-bf0e-606fdb336f51\") "
Sep 17 21:04:27.695504 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.695406    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/97bc6251-5ff3-4395-bf0e-606fdb336f51-kube-rbac-proxy-sar-config\") pod \"97bc6251-5ff3-4395-bf0e-606fdb336f51\" (UID: \"97bc6251-5ff3-4395-bf0e-606fdb336f51\") "
Sep 17 21:04:27.695504 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.695437    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/97bc6251-5ff3-4395-bf0e-606fdb336f51-cabundle-cert\") pod \"97bc6251-5ff3-4395-bf0e-606fdb336f51\" (UID: \"97bc6251-5ff3-4395-bf0e-606fdb336f51\") "
Sep 17 21:04:27.695793 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.695749    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97bc6251-5ff3-4395-bf0e-606fdb336f51-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "97bc6251-5ff3-4395-bf0e-606fdb336f51" (UID: "97bc6251-5ff3-4395-bf0e-606fdb336f51"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 21:04:27.695869 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.695823    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97bc6251-5ff3-4395-bf0e-606fdb336f51-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "97bc6251-5ff3-4395-bf0e-606fdb336f51" (UID: "97bc6251-5ff3-4395-bf0e-606fdb336f51"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:04:27.695869 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.695852    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97bc6251-5ff3-4395-bf0e-606fdb336f51-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "97bc6251-5ff3-4395-bf0e-606fdb336f51" (UID: "97bc6251-5ff3-4395-bf0e-606fdb336f51"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:04:27.697855 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.697830    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97bc6251-5ff3-4395-bf0e-606fdb336f51-kube-api-access-dnzt7" (OuterVolumeSpecName: "kube-api-access-dnzt7") pod "97bc6251-5ff3-4395-bf0e-606fdb336f51" (UID: "97bc6251-5ff3-4395-bf0e-606fdb336f51"). InnerVolumeSpecName "kube-api-access-dnzt7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:04:27.697953 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.697932    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97bc6251-5ff3-4395-bf0e-606fdb336f51-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "97bc6251-5ff3-4395-bf0e-606fdb336f51" (UID: "97bc6251-5ff3-4395-bf0e-606fdb336f51"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:04:27.764125 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.764092    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n"]
Sep 17 21:04:27.764495 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.764480    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97bc6251-5ff3-4395-bf0e-606fdb336f51" containerName="kube-rbac-proxy"
Sep 17 21:04:27.764545 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.764497    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="97bc6251-5ff3-4395-bf0e-606fdb336f51" containerName="kube-rbac-proxy"
Sep 17 21:04:27.764545 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.764507    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97bc6251-5ff3-4395-bf0e-606fdb336f51" containerName="storage-initializer"
Sep 17 21:04:27.764545 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.764515    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="97bc6251-5ff3-4395-bf0e-606fdb336f51" containerName="storage-initializer"
Sep 17 21:04:27.764545 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.764534    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97bc6251-5ff3-4395-bf0e-606fdb336f51" containerName="kserve-container"
Sep 17 21:04:27.764545 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.764540    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="97bc6251-5ff3-4395-bf0e-606fdb336f51" containerName="kserve-container"
Sep 17 21:04:27.764707 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.764595    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="97bc6251-5ff3-4395-bf0e-606fdb336f51" containerName="kserve-container"
Sep 17 21:04:27.764707 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.764611    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="97bc6251-5ff3-4395-bf0e-606fdb336f51" containerName="kube-rbac-proxy"
Sep 17 21:04:27.768001 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.767987    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n"
Sep 17 21:04:27.773186 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.773154    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-stop-predictor-serving-cert\""
Sep 17 21:04:27.779263 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.779242    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-stop-kube-rbac-proxy-sar-config\""
Sep 17 21:04:27.787255 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.787231    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n"]
Sep 17 21:04:27.795976 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.795955    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/97bc6251-5ff3-4395-bf0e-606fdb336f51-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:04:27.796071 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.795980    2567 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/97bc6251-5ff3-4395-bf0e-606fdb336f51-cabundle-cert\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:04:27.796071 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.795992    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/97bc6251-5ff3-4395-bf0e-606fdb336f51-proxy-tls\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:04:27.796071 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.796000    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dnzt7\" (UniqueName: \"kubernetes.io/projected/97bc6251-5ff3-4395-bf0e-606fdb336f51-kube-api-access-dnzt7\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:04:27.796071 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.796010    2567 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/97bc6251-5ff3-4395-bf0e-606fdb336f51-kserve-provision-location\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:04:27.810617 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.810589    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:04:27.821677 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.821653    2567 generic.go:356] "Generic (PLEG): container finished" podID="97bc6251-5ff3-4395-bf0e-606fdb336f51" containerID="9fac5e0ec96d46b5c92f71ebcbe8589e2279ca930a48fafa48a645a072d8e4e5" exitCode=0
Sep 17 21:04:27.821784 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.821684    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp" event={"ID":"97bc6251-5ff3-4395-bf0e-606fdb336f51","Type":"ContainerDied","Data":"9fac5e0ec96d46b5c92f71ebcbe8589e2279ca930a48fafa48a645a072d8e4e5"}
Sep 17 21:04:27.821784 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.821710    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp"
Sep 17 21:04:27.821784 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.821730    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp" event={"ID":"97bc6251-5ff3-4395-bf0e-606fdb336f51","Type":"ContainerDied","Data":"92741fc91c6d0ec02bcd63ff5d8bbac8d2846ab91c88caf2d8578f3946fc253f"}
Sep 17 21:04:27.821784 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.821769    2567 scope.go:117] "RemoveContainer" containerID="4c225c5dfaaef669137096787bd07da804ea8ecf5f5e8f6ff12d210243000c6c"
Sep 17 21:04:27.830480 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.830458    2567 scope.go:117] "RemoveContainer" containerID="9fac5e0ec96d46b5c92f71ebcbe8589e2279ca930a48fafa48a645a072d8e4e5"
Sep 17 21:04:27.837922 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.837901    2567 scope.go:117] "RemoveContainer" containerID="f458f1f2866ca44e816ab156a709bdd79b4ce4178d3095d2f8d010d7075412e3"
Sep 17 21:04:27.845350 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.845331    2567 scope.go:117] "RemoveContainer" containerID="4c225c5dfaaef669137096787bd07da804ea8ecf5f5e8f6ff12d210243000c6c"
Sep 17 21:04:27.845603 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:04:27.845586    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c225c5dfaaef669137096787bd07da804ea8ecf5f5e8f6ff12d210243000c6c\": container with ID starting with 4c225c5dfaaef669137096787bd07da804ea8ecf5f5e8f6ff12d210243000c6c not found: ID does not exist" containerID="4c225c5dfaaef669137096787bd07da804ea8ecf5f5e8f6ff12d210243000c6c"
Sep 17 21:04:27.845671 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.845611    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c225c5dfaaef669137096787bd07da804ea8ecf5f5e8f6ff12d210243000c6c"} err="failed to get container status \"4c225c5dfaaef669137096787bd07da804ea8ecf5f5e8f6ff12d210243000c6c\": rpc error: code = NotFound desc = could not find container \"4c225c5dfaaef669137096787bd07da804ea8ecf5f5e8f6ff12d210243000c6c\": container with ID starting with 4c225c5dfaaef669137096787bd07da804ea8ecf5f5e8f6ff12d210243000c6c not found: ID does not exist"
Sep 17 21:04:27.845671 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.845635    2567 scope.go:117] "RemoveContainer" containerID="9fac5e0ec96d46b5c92f71ebcbe8589e2279ca930a48fafa48a645a072d8e4e5"
Sep 17 21:04:27.845986 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:04:27.845960    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fac5e0ec96d46b5c92f71ebcbe8589e2279ca930a48fafa48a645a072d8e4e5\": container with ID starting with 9fac5e0ec96d46b5c92f71ebcbe8589e2279ca930a48fafa48a645a072d8e4e5 not found: ID does not exist" containerID="9fac5e0ec96d46b5c92f71ebcbe8589e2279ca930a48fafa48a645a072d8e4e5"
Sep 17 21:04:27.846077 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.845995    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fac5e0ec96d46b5c92f71ebcbe8589e2279ca930a48fafa48a645a072d8e4e5"} err="failed to get container status \"9fac5e0ec96d46b5c92f71ebcbe8589e2279ca930a48fafa48a645a072d8e4e5\": rpc error: code = NotFound desc = could not find container \"9fac5e0ec96d46b5c92f71ebcbe8589e2279ca930a48fafa48a645a072d8e4e5\": container with ID starting with 9fac5e0ec96d46b5c92f71ebcbe8589e2279ca930a48fafa48a645a072d8e4e5 not found: ID does not exist"
Sep 17 21:04:27.846077 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.846017    2567 scope.go:117] "RemoveContainer" containerID="f458f1f2866ca44e816ab156a709bdd79b4ce4178d3095d2f8d010d7075412e3"
Sep 17 21:04:27.846437 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:04:27.846400    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f458f1f2866ca44e816ab156a709bdd79b4ce4178d3095d2f8d010d7075412e3\": container with ID starting with f458f1f2866ca44e816ab156a709bdd79b4ce4178d3095d2f8d010d7075412e3 not found: ID does not exist" containerID="f458f1f2866ca44e816ab156a709bdd79b4ce4178d3095d2f8d010d7075412e3"
Sep 17 21:04:27.846586 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.846443    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f458f1f2866ca44e816ab156a709bdd79b4ce4178d3095d2f8d010d7075412e3"} err="failed to get container status \"f458f1f2866ca44e816ab156a709bdd79b4ce4178d3095d2f8d010d7075412e3\": rpc error: code = NotFound desc = could not find container \"f458f1f2866ca44e816ab156a709bdd79b4ce4178d3095d2f8d010d7075412e3\": container with ID starting with f458f1f2866ca44e816ab156a709bdd79b4ce4178d3095d2f8d010d7075412e3 not found: ID does not exist"
Sep 17 21:04:27.882776 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.882690    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp"]
Sep 17 21:04:27.897348 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.897322    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vpm9\" (UniqueName: \"kubernetes.io/projected/395c4884-f6d1-450a-9db9-1073b32e5a53-kube-api-access-5vpm9\") pod \"isvc-canary-stop-predictor-55656fdf77-5bm4n\" (UID: \"395c4884-f6d1-450a-9db9-1073b32e5a53\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n"
Sep 17 21:04:27.897486 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.897372    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/395c4884-f6d1-450a-9db9-1073b32e5a53-kserve-provision-location\") pod \"isvc-canary-stop-predictor-55656fdf77-5bm4n\" (UID: \"395c4884-f6d1-450a-9db9-1073b32e5a53\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n"
Sep 17 21:04:27.897486 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.897399    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/395c4884-f6d1-450a-9db9-1073b32e5a53-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-55656fdf77-5bm4n\" (UID: \"395c4884-f6d1-450a-9db9-1073b32e5a53\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n"
Sep 17 21:04:27.897578 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.897501    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/395c4884-f6d1-450a-9db9-1073b32e5a53-proxy-tls\") pod \"isvc-canary-stop-predictor-55656fdf77-5bm4n\" (UID: \"395c4884-f6d1-450a-9db9-1073b32e5a53\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n"
Sep 17 21:04:27.897578 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.897525    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/395c4884-f6d1-450a-9db9-1073b32e5a53-cabundle-cert\") pod \"isvc-canary-stop-predictor-55656fdf77-5bm4n\" (UID: \"395c4884-f6d1-450a-9db9-1073b32e5a53\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n"
Sep 17 21:04:27.902315 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.902291    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-97977b6b5-cp7gp"]
Sep 17 21:04:27.962388 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.962348    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:04:27.998179 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.998145    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/395c4884-f6d1-450a-9db9-1073b32e5a53-proxy-tls\") pod \"isvc-canary-stop-predictor-55656fdf77-5bm4n\" (UID: \"395c4884-f6d1-450a-9db9-1073b32e5a53\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n"
Sep 17 21:04:27.998179 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.998177    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/395c4884-f6d1-450a-9db9-1073b32e5a53-cabundle-cert\") pod \"isvc-canary-stop-predictor-55656fdf77-5bm4n\" (UID: \"395c4884-f6d1-450a-9db9-1073b32e5a53\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n"
Sep 17 21:04:27.998428 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.998229    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vpm9\" (UniqueName: \"kubernetes.io/projected/395c4884-f6d1-450a-9db9-1073b32e5a53-kube-api-access-5vpm9\") pod \"isvc-canary-stop-predictor-55656fdf77-5bm4n\" (UID: \"395c4884-f6d1-450a-9db9-1073b32e5a53\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n"
Sep 17 21:04:27.998428 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.998269    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/395c4884-f6d1-450a-9db9-1073b32e5a53-kserve-provision-location\") pod \"isvc-canary-stop-predictor-55656fdf77-5bm4n\" (UID: \"395c4884-f6d1-450a-9db9-1073b32e5a53\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n"
Sep 17 21:04:27.998428 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.998311    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/395c4884-f6d1-450a-9db9-1073b32e5a53-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-55656fdf77-5bm4n\" (UID: \"395c4884-f6d1-450a-9db9-1073b32e5a53\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n"
Sep 17 21:04:27.998830 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.998803    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/395c4884-f6d1-450a-9db9-1073b32e5a53-kserve-provision-location\") pod \"isvc-canary-stop-predictor-55656fdf77-5bm4n\" (UID: \"395c4884-f6d1-450a-9db9-1073b32e5a53\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n"
Sep 17 21:04:27.999183 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.999159    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/395c4884-f6d1-450a-9db9-1073b32e5a53-cabundle-cert\") pod \"isvc-canary-stop-predictor-55656fdf77-5bm4n\" (UID: \"395c4884-f6d1-450a-9db9-1073b32e5a53\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n"
Sep 17 21:04:27.999261 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:27.999159    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/395c4884-f6d1-450a-9db9-1073b32e5a53-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-55656fdf77-5bm4n\" (UID: \"395c4884-f6d1-450a-9db9-1073b32e5a53\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n"
Sep 17 21:04:28.000742 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:28.000724    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/395c4884-f6d1-450a-9db9-1073b32e5a53-proxy-tls\") pod \"isvc-canary-stop-predictor-55656fdf77-5bm4n\" (UID: \"395c4884-f6d1-450a-9db9-1073b32e5a53\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n"
Sep 17 21:04:28.010258 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:28.010235    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vpm9\" (UniqueName: \"kubernetes.io/projected/395c4884-f6d1-450a-9db9-1073b32e5a53-kube-api-access-5vpm9\") pod \"isvc-canary-stop-predictor-55656fdf77-5bm4n\" (UID: \"395c4884-f6d1-450a-9db9-1073b32e5a53\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n"
Sep 17 21:04:28.078251 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:28.078214    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n"
Sep 17 21:04:28.114539 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:28.114488    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:04:28.169634 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:28.169556    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97bc6251-5ff3-4395-bf0e-606fdb336f51" path="/var/lib/kubelet/pods/97bc6251-5ff3-4395-bf0e-606fdb336f51/volumes"
Sep 17 21:04:28.365020 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:28.364993    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n"]
Sep 17 21:04:28.367671 ip-10-0-132-201 kubenswrapper[2567]: W0917 21:04:28.367646    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod395c4884_f6d1_450a_9db9_1073b32e5a53.slice/crio-d23e7ab38d8b70498bddc72021e1ef7a3d645d7edfc37147b17901db0339c08c WatchSource:0}: Error finding container d23e7ab38d8b70498bddc72021e1ef7a3d645d7edfc37147b17901db0339c08c: Status 404 returned error can't find the container with id d23e7ab38d8b70498bddc72021e1ef7a3d645d7edfc37147b17901db0339c08c
Sep 17 21:04:28.828924 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:28.828886    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n" event={"ID":"395c4884-f6d1-450a-9db9-1073b32e5a53","Type":"ContainerStarted","Data":"3c72fb9c99e4a6bf0d82633141de22a5c3f32207348f3b05b91fb8a1452fc27c"}
Sep 17 21:04:28.829318 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:28.828958    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n" event={"ID":"395c4884-f6d1-450a-9db9-1073b32e5a53","Type":"ContainerStarted","Data":"d23e7ab38d8b70498bddc72021e1ef7a3d645d7edfc37147b17901db0339c08c"}
Sep 17 21:04:32.164612 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:32.164561    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h" podUID="e5f88102-8b3a-48d6-9f09-96aae9f59be9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Sep 17 21:04:32.608067 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:32.608027    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns" podUID="a7079774-6102-4c26-b948-55bdaf0bd825" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused"
Sep 17 21:04:32.850800 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:32.850750    2567 generic.go:356] "Generic (PLEG): container finished" podID="395c4884-f6d1-450a-9db9-1073b32e5a53" containerID="3c72fb9c99e4a6bf0d82633141de22a5c3f32207348f3b05b91fb8a1452fc27c" exitCode=0
Sep 17 21:04:32.850974 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:32.850824    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n" event={"ID":"395c4884-f6d1-450a-9db9-1073b32e5a53","Type":"ContainerDied","Data":"3c72fb9c99e4a6bf0d82633141de22a5c3f32207348f3b05b91fb8a1452fc27c"}
Sep 17 21:04:33.857468 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:33.857431    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n" event={"ID":"395c4884-f6d1-450a-9db9-1073b32e5a53","Type":"ContainerStarted","Data":"031f96e3de2a141caa660d3a0895ec41db7aae1b0f1f4b27c7a13949a07514d2"}
Sep 17 21:04:33.857872 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:33.857500    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n" event={"ID":"395c4884-f6d1-450a-9db9-1073b32e5a53","Type":"ContainerStarted","Data":"df9d0cc1d0429117a9097b645d93d4d21ce15f79dd9fb220ca5d1e08123d0038"}
Sep 17 21:04:33.857872 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:33.857778    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n"
Sep 17 21:04:33.857974 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:33.857924    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n"
Sep 17 21:04:33.859073 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:33.859050    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n" podUID="395c4884-f6d1-450a-9db9-1073b32e5a53" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Sep 17 21:04:33.903522 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:33.903466    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n" podStartSLOduration=3.018583198 podStartE2EDuration="6.903447429s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.884864231s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:04:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:04:33.902546883 +0000 UTC m=+938.248148707" watchObservedRunningTime="2026-09-17 21:04:33.903447429 +0000 UTC m=+938.249049253"
Sep 17 21:04:34.862583 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:34.862533    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n" podUID="395c4884-f6d1-450a-9db9-1073b32e5a53" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Sep 17 21:04:39.867700 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:39.867671    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n"
Sep 17 21:04:39.868213 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:39.868185    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n" podUID="395c4884-f6d1-450a-9db9-1073b32e5a53" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Sep 17 21:04:42.168809 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:42.168780    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h"
Sep 17 21:04:42.607873 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:42.607830    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns" podUID="a7079774-6102-4c26-b948-55bdaf0bd825" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused"
Sep 17 21:04:49.868669 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:49.868621    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n" podUID="395c4884-f6d1-450a-9db9-1073b32e5a53" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Sep 17 21:04:52.608409 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:52.608377    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns"
Sep 17 21:04:52.916199 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:52.916124    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:04:53.050622 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:53.050591    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:04:56.158522 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:56.158490    2567 scope.go:117] "RemoveContainer" containerID="03be52ac552bf7a273017c0ce4fa72dd079ecbeeeb1eb022ad2e213b05e78aa2"
Sep 17 21:04:59.868446 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:04:59.868407    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n" podUID="395c4884-f6d1-450a-9db9-1073b32e5a53" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Sep 17 21:05:02.997629 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:02.997578    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:05:03.072378 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:03.072342    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h"]
Sep 17 21:05:03.072640 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:03.072614    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h" podUID="e5f88102-8b3a-48d6-9f09-96aae9f59be9" containerName="kserve-container" containerID="cri-o://f9b2917588a2fb4ed6d4523ff108a4532ca0d3089edc0859a08f7c482697eeb8" gracePeriod=30
Sep 17 21:05:03.072746 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:03.072708    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h" podUID="e5f88102-8b3a-48d6-9f09-96aae9f59be9" containerName="kube-rbac-proxy" containerID="cri-o://a0215c4c0c11c4ae7231a34ce6e413cf98a4e04e0dbbcae21dd792953393d865" gracePeriod=30
Sep 17 21:05:03.110299 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:03.110258    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:05:03.113944 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:03.113916    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns"]
Sep 17 21:05:03.114205 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:03.114183    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns" podUID="a7079774-6102-4c26-b948-55bdaf0bd825" containerName="kserve-container" containerID="cri-o://9d72fa55fea2305eb398278d067991dd019a4e4f4aa3419914f05e400bfcc0cd" gracePeriod=30
Sep 17 21:05:03.114334 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:03.114260    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns" podUID="a7079774-6102-4c26-b948-55bdaf0bd825" containerName="kube-rbac-proxy" containerID="cri-o://c88022be5f3a814ea5c93e18d5fef48906da7125b3c9468a4e19d66473cfa243" gracePeriod=30
Sep 17 21:05:03.197104 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:03.197074    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:05:03.296915 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:03.296835    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:05:03.370394 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:03.370349    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:05:03.646017 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:03.645924    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:05:03.997869 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:03.997841    2567 generic.go:356] "Generic (PLEG): container finished" podID="a7079774-6102-4c26-b948-55bdaf0bd825" containerID="c88022be5f3a814ea5c93e18d5fef48906da7125b3c9468a4e19d66473cfa243" exitCode=2
Sep 17 21:05:03.998293 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:03.997910    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns" event={"ID":"a7079774-6102-4c26-b948-55bdaf0bd825","Type":"ContainerDied","Data":"c88022be5f3a814ea5c93e18d5fef48906da7125b3c9468a4e19d66473cfa243"}
Sep 17 21:05:04.000174 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:04.000153    2567 generic.go:356] "Generic (PLEG): container finished" podID="e5f88102-8b3a-48d6-9f09-96aae9f59be9" containerID="a0215c4c0c11c4ae7231a34ce6e413cf98a4e04e0dbbcae21dd792953393d865" exitCode=2
Sep 17 21:05:04.000275 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:04.000194    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h" event={"ID":"e5f88102-8b3a-48d6-9f09-96aae9f59be9","Type":"ContainerDied","Data":"a0215c4c0c11c4ae7231a34ce6e413cf98a4e04e0dbbcae21dd792953393d865"}
Sep 17 21:05:05.505636 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:05.505595    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h" podUID="e5f88102-8b3a-48d6-9f09-96aae9f59be9" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.26:8643/healthz\": dial tcp 10.132.0.26:8643: connect: connection refused"
Sep 17 21:05:07.364943 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:07.364915    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns"
Sep 17 21:05:07.414729 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:07.414642    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a7079774-6102-4c26-b948-55bdaf0bd825-kserve-provision-location\") pod \"a7079774-6102-4c26-b948-55bdaf0bd825\" (UID: \"a7079774-6102-4c26-b948-55bdaf0bd825\") "
Sep 17 21:05:07.414729 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:07.414704    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a7079774-6102-4c26-b948-55bdaf0bd825-proxy-tls\") pod \"a7079774-6102-4c26-b948-55bdaf0bd825\" (UID: \"a7079774-6102-4c26-b948-55bdaf0bd825\") "
Sep 17 21:05:07.414973 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:07.414892    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a7079774-6102-4c26-b948-55bdaf0bd825-cabundle-cert\") pod \"a7079774-6102-4c26-b948-55bdaf0bd825\" (UID: \"a7079774-6102-4c26-b948-55bdaf0bd825\") "
Sep 17 21:05:07.414973 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:07.414924    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlpjs\" (UniqueName: \"kubernetes.io/projected/a7079774-6102-4c26-b948-55bdaf0bd825-kube-api-access-tlpjs\") pod \"a7079774-6102-4c26-b948-55bdaf0bd825\" (UID: \"a7079774-6102-4c26-b948-55bdaf0bd825\") "
Sep 17 21:05:07.414973 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:07.414929    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7079774-6102-4c26-b948-55bdaf0bd825-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "a7079774-6102-4c26-b948-55bdaf0bd825" (UID: "a7079774-6102-4c26-b948-55bdaf0bd825"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 21:05:07.414973 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:07.414954    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a7079774-6102-4c26-b948-55bdaf0bd825-kube-rbac-proxy-sar-config\") pod \"a7079774-6102-4c26-b948-55bdaf0bd825\" (UID: \"a7079774-6102-4c26-b948-55bdaf0bd825\") "
Sep 17 21:05:07.415212 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:07.415197    2567 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a7079774-6102-4c26-b948-55bdaf0bd825-kserve-provision-location\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:05:07.415273 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:07.415252    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7079774-6102-4c26-b948-55bdaf0bd825-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "a7079774-6102-4c26-b948-55bdaf0bd825" (UID: "a7079774-6102-4c26-b948-55bdaf0bd825"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:05:07.415317 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:07.415263    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7079774-6102-4c26-b948-55bdaf0bd825-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "a7079774-6102-4c26-b948-55bdaf0bd825" (UID: "a7079774-6102-4c26-b948-55bdaf0bd825"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:05:07.416649 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:07.416626    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7079774-6102-4c26-b948-55bdaf0bd825-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "a7079774-6102-4c26-b948-55bdaf0bd825" (UID: "a7079774-6102-4c26-b948-55bdaf0bd825"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:05:07.416967 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:07.416942    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7079774-6102-4c26-b948-55bdaf0bd825-kube-api-access-tlpjs" (OuterVolumeSpecName: "kube-api-access-tlpjs") pod "a7079774-6102-4c26-b948-55bdaf0bd825" (UID: "a7079774-6102-4c26-b948-55bdaf0bd825"). InnerVolumeSpecName "kube-api-access-tlpjs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:05:07.515612 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:07.515573    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a7079774-6102-4c26-b948-55bdaf0bd825-proxy-tls\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:05:07.515612 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:07.515604    2567 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a7079774-6102-4c26-b948-55bdaf0bd825-cabundle-cert\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:05:07.515612 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:07.515613    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlpjs\" (UniqueName: \"kubernetes.io/projected/a7079774-6102-4c26-b948-55bdaf0bd825-kube-api-access-tlpjs\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:05:07.515612 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:07.515622    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a7079774-6102-4c26-b948-55bdaf0bd825-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:05:08.021214 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.021183    2567 generic.go:356] "Generic (PLEG): container finished" podID="e5f88102-8b3a-48d6-9f09-96aae9f59be9" containerID="f9b2917588a2fb4ed6d4523ff108a4532ca0d3089edc0859a08f7c482697eeb8" exitCode=0
Sep 17 21:05:08.021386 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.021246    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h" event={"ID":"e5f88102-8b3a-48d6-9f09-96aae9f59be9","Type":"ContainerDied","Data":"f9b2917588a2fb4ed6d4523ff108a4532ca0d3089edc0859a08f7c482697eeb8"}
Sep 17 21:05:08.023255 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.023232    2567 generic.go:356] "Generic (PLEG): container finished" podID="a7079774-6102-4c26-b948-55bdaf0bd825" containerID="9d72fa55fea2305eb398278d067991dd019a4e4f4aa3419914f05e400bfcc0cd" exitCode=0
Sep 17 21:05:08.023381 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.023290    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns"
Sep 17 21:05:08.023381 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.023300    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns" event={"ID":"a7079774-6102-4c26-b948-55bdaf0bd825","Type":"ContainerDied","Data":"9d72fa55fea2305eb398278d067991dd019a4e4f4aa3419914f05e400bfcc0cd"}
Sep 17 21:05:08.023381 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.023342    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns" event={"ID":"a7079774-6102-4c26-b948-55bdaf0bd825","Type":"ContainerDied","Data":"12e5e51b5819a91057717c63aef2a7cc38ba6fa3187a812c404c69100e651628"}
Sep 17 21:05:08.023381 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.023368    2567 scope.go:117] "RemoveContainer" containerID="c88022be5f3a814ea5c93e18d5fef48906da7125b3c9468a4e19d66473cfa243"
Sep 17 21:05:08.032480 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.032461    2567 scope.go:117] "RemoveContainer" containerID="9d72fa55fea2305eb398278d067991dd019a4e4f4aa3419914f05e400bfcc0cd"
Sep 17 21:05:08.041408 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.041391    2567 scope.go:117] "RemoveContainer" containerID="1107586fab4aa59c3a77fc0591771b931777afee9945a9484c1468246d3ef0a7"
Sep 17 21:05:08.048788 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.048750    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns"]
Sep 17 21:05:08.051369 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.051346    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-669c3-predictor-59fd788c85-7lqns"]
Sep 17 21:05:08.052566 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.052547    2567 scope.go:117] "RemoveContainer" containerID="c88022be5f3a814ea5c93e18d5fef48906da7125b3c9468a4e19d66473cfa243"
Sep 17 21:05:08.052908 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:05:08.052887    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c88022be5f3a814ea5c93e18d5fef48906da7125b3c9468a4e19d66473cfa243\": container with ID starting with c88022be5f3a814ea5c93e18d5fef48906da7125b3c9468a4e19d66473cfa243 not found: ID does not exist" containerID="c88022be5f3a814ea5c93e18d5fef48906da7125b3c9468a4e19d66473cfa243"
Sep 17 21:05:08.052970 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.052921    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c88022be5f3a814ea5c93e18d5fef48906da7125b3c9468a4e19d66473cfa243"} err="failed to get container status \"c88022be5f3a814ea5c93e18d5fef48906da7125b3c9468a4e19d66473cfa243\": rpc error: code = NotFound desc = could not find container \"c88022be5f3a814ea5c93e18d5fef48906da7125b3c9468a4e19d66473cfa243\": container with ID starting with c88022be5f3a814ea5c93e18d5fef48906da7125b3c9468a4e19d66473cfa243 not found: ID does not exist"
Sep 17 21:05:08.052970 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.052946    2567 scope.go:117] "RemoveContainer" containerID="9d72fa55fea2305eb398278d067991dd019a4e4f4aa3419914f05e400bfcc0cd"
Sep 17 21:05:08.053248 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:05:08.053228    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d72fa55fea2305eb398278d067991dd019a4e4f4aa3419914f05e400bfcc0cd\": container with ID starting with 9d72fa55fea2305eb398278d067991dd019a4e4f4aa3419914f05e400bfcc0cd not found: ID does not exist" containerID="9d72fa55fea2305eb398278d067991dd019a4e4f4aa3419914f05e400bfcc0cd"
Sep 17 21:05:08.053310 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.053253    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d72fa55fea2305eb398278d067991dd019a4e4f4aa3419914f05e400bfcc0cd"} err="failed to get container status \"9d72fa55fea2305eb398278d067991dd019a4e4f4aa3419914f05e400bfcc0cd\": rpc error: code = NotFound desc = could not find container \"9d72fa55fea2305eb398278d067991dd019a4e4f4aa3419914f05e400bfcc0cd\": container with ID starting with 9d72fa55fea2305eb398278d067991dd019a4e4f4aa3419914f05e400bfcc0cd not found: ID does not exist"
Sep 17 21:05:08.053310 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.053269    2567 scope.go:117] "RemoveContainer" containerID="1107586fab4aa59c3a77fc0591771b931777afee9945a9484c1468246d3ef0a7"
Sep 17 21:05:08.053595 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:05:08.053581    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1107586fab4aa59c3a77fc0591771b931777afee9945a9484c1468246d3ef0a7\": container with ID starting with 1107586fab4aa59c3a77fc0591771b931777afee9945a9484c1468246d3ef0a7 not found: ID does not exist" containerID="1107586fab4aa59c3a77fc0591771b931777afee9945a9484c1468246d3ef0a7"
Sep 17 21:05:08.053644 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.053599    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1107586fab4aa59c3a77fc0591771b931777afee9945a9484c1468246d3ef0a7"} err="failed to get container status \"1107586fab4aa59c3a77fc0591771b931777afee9945a9484c1468246d3ef0a7\": rpc error: code = NotFound desc = could not find container \"1107586fab4aa59c3a77fc0591771b931777afee9945a9484c1468246d3ef0a7\": container with ID starting with 1107586fab4aa59c3a77fc0591771b931777afee9945a9484c1468246d3ef0a7 not found: ID does not exist"
Sep 17 21:05:08.108837 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.108812    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h"
Sep 17 21:05:08.170505 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.170472    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7079774-6102-4c26-b948-55bdaf0bd825" path="/var/lib/kubelet/pods/a7079774-6102-4c26-b948-55bdaf0bd825/volumes"
Sep 17 21:05:08.220720 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.220697    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e5f88102-8b3a-48d6-9f09-96aae9f59be9-kserve-provision-location\") pod \"e5f88102-8b3a-48d6-9f09-96aae9f59be9\" (UID: \"e5f88102-8b3a-48d6-9f09-96aae9f59be9\") "
Sep 17 21:05:08.220892 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.220743    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6hkm\" (UniqueName: \"kubernetes.io/projected/e5f88102-8b3a-48d6-9f09-96aae9f59be9-kube-api-access-t6hkm\") pod \"e5f88102-8b3a-48d6-9f09-96aae9f59be9\" (UID: \"e5f88102-8b3a-48d6-9f09-96aae9f59be9\") "
Sep 17 21:05:08.220892 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.220795    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e5f88102-8b3a-48d6-9f09-96aae9f59be9-kube-rbac-proxy-sar-config\") pod \"e5f88102-8b3a-48d6-9f09-96aae9f59be9\" (UID: \"e5f88102-8b3a-48d6-9f09-96aae9f59be9\") "
Sep 17 21:05:08.220892 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.220818    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e5f88102-8b3a-48d6-9f09-96aae9f59be9-cabundle-cert\") pod \"e5f88102-8b3a-48d6-9f09-96aae9f59be9\" (UID: \"e5f88102-8b3a-48d6-9f09-96aae9f59be9\") "
Sep 17 21:05:08.220892 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.220846    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e5f88102-8b3a-48d6-9f09-96aae9f59be9-proxy-tls\") pod \"e5f88102-8b3a-48d6-9f09-96aae9f59be9\" (UID: \"e5f88102-8b3a-48d6-9f09-96aae9f59be9\") "
Sep 17 21:05:08.221058 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.221026    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5f88102-8b3a-48d6-9f09-96aae9f59be9-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "e5f88102-8b3a-48d6-9f09-96aae9f59be9" (UID: "e5f88102-8b3a-48d6-9f09-96aae9f59be9"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 21:05:08.221202 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.221166    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5f88102-8b3a-48d6-9f09-96aae9f59be9-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "e5f88102-8b3a-48d6-9f09-96aae9f59be9" (UID: "e5f88102-8b3a-48d6-9f09-96aae9f59be9"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:05:08.221202 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.221193    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5f88102-8b3a-48d6-9f09-96aae9f59be9-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "e5f88102-8b3a-48d6-9f09-96aae9f59be9" (UID: "e5f88102-8b3a-48d6-9f09-96aae9f59be9"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:05:08.222943 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.222924    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5f88102-8b3a-48d6-9f09-96aae9f59be9-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "e5f88102-8b3a-48d6-9f09-96aae9f59be9" (UID: "e5f88102-8b3a-48d6-9f09-96aae9f59be9"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:05:08.222943 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.222934    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5f88102-8b3a-48d6-9f09-96aae9f59be9-kube-api-access-t6hkm" (OuterVolumeSpecName: "kube-api-access-t6hkm") pod "e5f88102-8b3a-48d6-9f09-96aae9f59be9" (UID: "e5f88102-8b3a-48d6-9f09-96aae9f59be9"). InnerVolumeSpecName "kube-api-access-t6hkm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:05:08.321927 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.321871    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e5f88102-8b3a-48d6-9f09-96aae9f59be9-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:05:08.321927 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.321905    2567 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e5f88102-8b3a-48d6-9f09-96aae9f59be9-cabundle-cert\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:05:08.321927 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.321917    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e5f88102-8b3a-48d6-9f09-96aae9f59be9-proxy-tls\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:05:08.321927 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.321925    2567 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e5f88102-8b3a-48d6-9f09-96aae9f59be9-kserve-provision-location\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:05:08.321927 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:08.321934    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6hkm\" (UniqueName: \"kubernetes.io/projected/e5f88102-8b3a-48d6-9f09-96aae9f59be9-kube-api-access-t6hkm\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:05:09.029464 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:09.029428    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h"
Sep 17 21:05:09.029464 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:09.029439    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h" event={"ID":"e5f88102-8b3a-48d6-9f09-96aae9f59be9","Type":"ContainerDied","Data":"50a8c9a578ac9c56daea6335af403f6c84b57420eddbc7080e1efb1944ff8542"}
Sep 17 21:05:09.030001 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:09.029515    2567 scope.go:117] "RemoveContainer" containerID="a0215c4c0c11c4ae7231a34ce6e413cf98a4e04e0dbbcae21dd792953393d865"
Sep 17 21:05:09.037840 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:09.037818    2567 scope.go:117] "RemoveContainer" containerID="f9b2917588a2fb4ed6d4523ff108a4532ca0d3089edc0859a08f7c482697eeb8"
Sep 17 21:05:09.044983 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:09.044964    2567 scope.go:117] "RemoveContainer" containerID="dda92f4aef5340bb644e5e8cf59ba3308a3c7947b4b869f864c4f00de2d3ea6c"
Sep 17 21:05:09.051514 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:09.051489    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h"]
Sep 17 21:05:09.055889 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:09.055867    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-669c3-predictor-b5f549468-wfk6h"]
Sep 17 21:05:09.869132 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:09.869096    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n" podUID="395c4884-f6d1-450a-9db9-1073b32e5a53" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Sep 17 21:05:10.170045 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:10.169970    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5f88102-8b3a-48d6-9f09-96aae9f59be9" path="/var/lib/kubelet/pods/e5f88102-8b3a-48d6-9f09-96aae9f59be9/volumes"
Sep 17 21:05:19.869122 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:19.869034    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n" podUID="395c4884-f6d1-450a-9db9-1073b32e5a53" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Sep 17 21:05:29.868988 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:29.868943    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n" podUID="395c4884-f6d1-450a-9db9-1073b32e5a53" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Sep 17 21:05:30.258311 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:30.258272    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:05:30.323398 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:30.323360    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:05:39.868540 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:39.868500    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n" podUID="395c4884-f6d1-450a-9db9-1073b32e5a53" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Sep 17 21:05:44.165180 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:44.165134    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n" podUID="395c4884-f6d1-450a-9db9-1073b32e5a53" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Sep 17 21:05:50.606435 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:50.606370    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:05:50.707559 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:50.707521    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:05:54.168927 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:05:54.168894    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n"
Sep 17 21:06:07.568130 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:07.568071    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:06:07.581077 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:07.581046    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n"]
Sep 17 21:06:07.581312 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:07.581285    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n" podUID="395c4884-f6d1-450a-9db9-1073b32e5a53" containerName="kserve-container" containerID="cri-o://df9d0cc1d0429117a9097b645d93d4d21ce15f79dd9fb220ca5d1e08123d0038" gracePeriod=30
Sep 17 21:06:07.581389 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:07.581343    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n" podUID="395c4884-f6d1-450a-9db9-1073b32e5a53" containerName="kube-rbac-proxy" containerID="cri-o://031f96e3de2a141caa660d3a0895ec41db7aae1b0f1f4b27c7a13949a07514d2" gracePeriod=30
Sep 17 21:06:07.632257 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:07.632218    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:06:08.275955 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:08.275923    2567 generic.go:356] "Generic (PLEG): container finished" podID="395c4884-f6d1-450a-9db9-1073b32e5a53" containerID="031f96e3de2a141caa660d3a0895ec41db7aae1b0f1f4b27c7a13949a07514d2" exitCode=2
Sep 17 21:06:08.276122 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:08.275961    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n" event={"ID":"395c4884-f6d1-450a-9db9-1073b32e5a53","Type":"ContainerDied","Data":"031f96e3de2a141caa660d3a0895ec41db7aae1b0f1f4b27c7a13949a07514d2"}
Sep 17 21:06:09.862913 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:09.862869    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n" podUID="395c4884-f6d1-450a-9db9-1073b32e5a53" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.28:8643/healthz\": dial tcp 10.132.0.28:8643: connect: connection refused"
Sep 17 21:06:12.293256 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:12.293221    2567 generic.go:356] "Generic (PLEG): container finished" podID="395c4884-f6d1-450a-9db9-1073b32e5a53" containerID="df9d0cc1d0429117a9097b645d93d4d21ce15f79dd9fb220ca5d1e08123d0038" exitCode=0
Sep 17 21:06:12.293685 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:12.293291    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n" event={"ID":"395c4884-f6d1-450a-9db9-1073b32e5a53","Type":"ContainerDied","Data":"df9d0cc1d0429117a9097b645d93d4d21ce15f79dd9fb220ca5d1e08123d0038"}
Sep 17 21:06:12.325950 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:12.325927    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n"
Sep 17 21:06:12.425337 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:12.425267    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vpm9\" (UniqueName: \"kubernetes.io/projected/395c4884-f6d1-450a-9db9-1073b32e5a53-kube-api-access-5vpm9\") pod \"395c4884-f6d1-450a-9db9-1073b32e5a53\" (UID: \"395c4884-f6d1-450a-9db9-1073b32e5a53\") "
Sep 17 21:06:12.425337 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:12.425303    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/395c4884-f6d1-450a-9db9-1073b32e5a53-cabundle-cert\") pod \"395c4884-f6d1-450a-9db9-1073b32e5a53\" (UID: \"395c4884-f6d1-450a-9db9-1073b32e5a53\") "
Sep 17 21:06:12.425337 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:12.425331    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/395c4884-f6d1-450a-9db9-1073b32e5a53-kserve-provision-location\") pod \"395c4884-f6d1-450a-9db9-1073b32e5a53\" (UID: \"395c4884-f6d1-450a-9db9-1073b32e5a53\") "
Sep 17 21:06:12.425544 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:12.425452    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/395c4884-f6d1-450a-9db9-1073b32e5a53-kube-rbac-proxy-sar-config\") pod \"395c4884-f6d1-450a-9db9-1073b32e5a53\" (UID: \"395c4884-f6d1-450a-9db9-1073b32e5a53\") "
Sep 17 21:06:12.425544 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:12.425495    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/395c4884-f6d1-450a-9db9-1073b32e5a53-proxy-tls\") pod \"395c4884-f6d1-450a-9db9-1073b32e5a53\" (UID: \"395c4884-f6d1-450a-9db9-1073b32e5a53\") "
Sep 17 21:06:12.425728 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:12.425640    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/395c4884-f6d1-450a-9db9-1073b32e5a53-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "395c4884-f6d1-450a-9db9-1073b32e5a53" (UID: "395c4884-f6d1-450a-9db9-1073b32e5a53"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 21:06:12.425876 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:12.425751    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/395c4884-f6d1-450a-9db9-1073b32e5a53-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "395c4884-f6d1-450a-9db9-1073b32e5a53" (UID: "395c4884-f6d1-450a-9db9-1073b32e5a53"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:06:12.425937 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:12.425877    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/395c4884-f6d1-450a-9db9-1073b32e5a53-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "395c4884-f6d1-450a-9db9-1073b32e5a53" (UID: "395c4884-f6d1-450a-9db9-1073b32e5a53"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:06:12.427263 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:12.427245    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/395c4884-f6d1-450a-9db9-1073b32e5a53-kube-api-access-5vpm9" (OuterVolumeSpecName: "kube-api-access-5vpm9") pod "395c4884-f6d1-450a-9db9-1073b32e5a53" (UID: "395c4884-f6d1-450a-9db9-1073b32e5a53"). InnerVolumeSpecName "kube-api-access-5vpm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:06:12.427549 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:12.427527    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/395c4884-f6d1-450a-9db9-1073b32e5a53-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "395c4884-f6d1-450a-9db9-1073b32e5a53" (UID: "395c4884-f6d1-450a-9db9-1073b32e5a53"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:06:12.527043 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:12.527007    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/395c4884-f6d1-450a-9db9-1073b32e5a53-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:06:12.527043 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:12.527037    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/395c4884-f6d1-450a-9db9-1073b32e5a53-proxy-tls\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:06:12.527043 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:12.527047    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vpm9\" (UniqueName: \"kubernetes.io/projected/395c4884-f6d1-450a-9db9-1073b32e5a53-kube-api-access-5vpm9\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:06:12.527329 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:12.527059    2567 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/395c4884-f6d1-450a-9db9-1073b32e5a53-cabundle-cert\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:06:12.527329 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:12.527068    2567 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/395c4884-f6d1-450a-9db9-1073b32e5a53-kserve-provision-location\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:06:13.304791 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:13.304741    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n" event={"ID":"395c4884-f6d1-450a-9db9-1073b32e5a53","Type":"ContainerDied","Data":"d23e7ab38d8b70498bddc72021e1ef7a3d645d7edfc37147b17901db0339c08c"}
Sep 17 21:06:13.305293 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:13.304839    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n"
Sep 17 21:06:13.305293 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:13.304843    2567 scope.go:117] "RemoveContainer" containerID="031f96e3de2a141caa660d3a0895ec41db7aae1b0f1f4b27c7a13949a07514d2"
Sep 17 21:06:13.313826 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:13.313807    2567 scope.go:117] "RemoveContainer" containerID="df9d0cc1d0429117a9097b645d93d4d21ce15f79dd9fb220ca5d1e08123d0038"
Sep 17 21:06:13.321144 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:13.321126    2567 scope.go:117] "RemoveContainer" containerID="3c72fb9c99e4a6bf0d82633141de22a5c3f32207348f3b05b91fb8a1452fc27c"
Sep 17 21:06:13.327047 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:13.326964    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n"]
Sep 17 21:06:13.328867 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:13.328841    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-predictor-55656fdf77-5bm4n"]
Sep 17 21:06:14.168978 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:14.168945    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="395c4884-f6d1-450a-9db9-1073b32e5a53" path="/var/lib/kubelet/pods/395c4884-f6d1-450a-9db9-1073b32e5a53/volumes"
Sep 17 21:06:17.795461 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.795424    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:06:17.834070 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.834035    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db"]
Sep 17 21:06:17.834372 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.834357    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5f88102-8b3a-48d6-9f09-96aae9f59be9" containerName="kserve-container"
Sep 17 21:06:17.834416 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.834376    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5f88102-8b3a-48d6-9f09-96aae9f59be9" containerName="kserve-container"
Sep 17 21:06:17.834416 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.834388    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="395c4884-f6d1-450a-9db9-1073b32e5a53" containerName="kube-rbac-proxy"
Sep 17 21:06:17.834416 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.834393    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="395c4884-f6d1-450a-9db9-1073b32e5a53" containerName="kube-rbac-proxy"
Sep 17 21:06:17.834416 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.834401    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="395c4884-f6d1-450a-9db9-1073b32e5a53" containerName="kserve-container"
Sep 17 21:06:17.834416 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.834407    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="395c4884-f6d1-450a-9db9-1073b32e5a53" containerName="kserve-container"
Sep 17 21:06:17.834588 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.834418    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5f88102-8b3a-48d6-9f09-96aae9f59be9" containerName="storage-initializer"
Sep 17 21:06:17.834588 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.834423    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5f88102-8b3a-48d6-9f09-96aae9f59be9" containerName="storage-initializer"
Sep 17 21:06:17.834588 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.834433    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7079774-6102-4c26-b948-55bdaf0bd825" containerName="kube-rbac-proxy"
Sep 17 21:06:17.834588 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.834442    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7079774-6102-4c26-b948-55bdaf0bd825" containerName="kube-rbac-proxy"
Sep 17 21:06:17.834588 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.834450    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="395c4884-f6d1-450a-9db9-1073b32e5a53" containerName="storage-initializer"
Sep 17 21:06:17.834588 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.834457    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="395c4884-f6d1-450a-9db9-1073b32e5a53" containerName="storage-initializer"
Sep 17 21:06:17.834588 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.834464    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5f88102-8b3a-48d6-9f09-96aae9f59be9" containerName="kube-rbac-proxy"
Sep 17 21:06:17.834588 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.834468    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5f88102-8b3a-48d6-9f09-96aae9f59be9" containerName="kube-rbac-proxy"
Sep 17 21:06:17.834588 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.834477    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7079774-6102-4c26-b948-55bdaf0bd825" containerName="storage-initializer"
Sep 17 21:06:17.834588 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.834482    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7079774-6102-4c26-b948-55bdaf0bd825" containerName="storage-initializer"
Sep 17 21:06:17.834588 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.834491    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7079774-6102-4c26-b948-55bdaf0bd825" containerName="kserve-container"
Sep 17 21:06:17.834588 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.834496    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7079774-6102-4c26-b948-55bdaf0bd825" containerName="kserve-container"
Sep 17 21:06:17.834588 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.834555    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5f88102-8b3a-48d6-9f09-96aae9f59be9" containerName="kserve-container"
Sep 17 21:06:17.834588 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.834563    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="395c4884-f6d1-450a-9db9-1073b32e5a53" containerName="kube-rbac-proxy"
Sep 17 21:06:17.834588 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.834570    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7079774-6102-4c26-b948-55bdaf0bd825" containerName="kube-rbac-proxy"
Sep 17 21:06:17.834588 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.834575    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7079774-6102-4c26-b948-55bdaf0bd825" containerName="kserve-container"
Sep 17 21:06:17.834588 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.834581    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="395c4884-f6d1-450a-9db9-1073b32e5a53" containerName="kserve-container"
Sep 17 21:06:17.834588 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.834587    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5f88102-8b3a-48d6-9f09-96aae9f59be9" containerName="kube-rbac-proxy"
Sep 17 21:06:17.839303 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.839286    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db"
Sep 17 21:06:17.842512 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.842492    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Sep 17 21:06:17.843067 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.843047    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Sep 17 21:06:17.843174 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.843153    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 17 21:06:17.843242 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.843162    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-primary-b61317-predictor-serving-cert\""
Sep 17 21:06:17.843242 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.843192    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 17 21:06:17.843671 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.843655    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-primary-b61317-kube-rbac-proxy-sar-config\""
Sep 17 21:06:17.843780 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.843692    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-5dj6n\""
Sep 17 21:06:17.848065 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.848043    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db"]
Sep 17 21:06:17.868608 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.868581    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfsdh\" (UniqueName: \"kubernetes.io/projected/92420c11-81b3-44aa-ac53-f139e0ae5329-kube-api-access-bfsdh\") pod \"isvc-primary-b61317-predictor-6d9bb74dbf-cb8db\" (UID: \"92420c11-81b3-44aa-ac53-f139e0ae5329\") " pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db"
Sep 17 21:06:17.868741 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.868614    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/92420c11-81b3-44aa-ac53-f139e0ae5329-kube-rbac-proxy-sar-config\") pod \"isvc-primary-b61317-predictor-6d9bb74dbf-cb8db\" (UID: \"92420c11-81b3-44aa-ac53-f139e0ae5329\") " pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db"
Sep 17 21:06:17.868741 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.868642    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/92420c11-81b3-44aa-ac53-f139e0ae5329-proxy-tls\") pod \"isvc-primary-b61317-predictor-6d9bb74dbf-cb8db\" (UID: \"92420c11-81b3-44aa-ac53-f139e0ae5329\") " pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db"
Sep 17 21:06:17.868851 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.868731    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/92420c11-81b3-44aa-ac53-f139e0ae5329-kserve-provision-location\") pod \"isvc-primary-b61317-predictor-6d9bb74dbf-cb8db\" (UID: \"92420c11-81b3-44aa-ac53-f139e0ae5329\") " pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db"
Sep 17 21:06:17.868851 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.868822    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/92420c11-81b3-44aa-ac53-f139e0ae5329-cabundle-cert\") pod \"isvc-primary-b61317-predictor-6d9bb74dbf-cb8db\" (UID: \"92420c11-81b3-44aa-ac53-f139e0ae5329\") " pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db"
Sep 17 21:06:17.970184 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.970139    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfsdh\" (UniqueName: \"kubernetes.io/projected/92420c11-81b3-44aa-ac53-f139e0ae5329-kube-api-access-bfsdh\") pod \"isvc-primary-b61317-predictor-6d9bb74dbf-cb8db\" (UID: \"92420c11-81b3-44aa-ac53-f139e0ae5329\") " pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db"
Sep 17 21:06:17.970184 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.970185    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/92420c11-81b3-44aa-ac53-f139e0ae5329-kube-rbac-proxy-sar-config\") pod \"isvc-primary-b61317-predictor-6d9bb74dbf-cb8db\" (UID: \"92420c11-81b3-44aa-ac53-f139e0ae5329\") " pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db"
Sep 17 21:06:17.970454 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.970216    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/92420c11-81b3-44aa-ac53-f139e0ae5329-proxy-tls\") pod \"isvc-primary-b61317-predictor-6d9bb74dbf-cb8db\" (UID: \"92420c11-81b3-44aa-ac53-f139e0ae5329\") " pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db"
Sep 17 21:06:17.970454 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.970243    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/92420c11-81b3-44aa-ac53-f139e0ae5329-kserve-provision-location\") pod \"isvc-primary-b61317-predictor-6d9bb74dbf-cb8db\" (UID: \"92420c11-81b3-44aa-ac53-f139e0ae5329\") " pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db"
Sep 17 21:06:17.970454 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.970274    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/92420c11-81b3-44aa-ac53-f139e0ae5329-cabundle-cert\") pod \"isvc-primary-b61317-predictor-6d9bb74dbf-cb8db\" (UID: \"92420c11-81b3-44aa-ac53-f139e0ae5329\") " pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db"
Sep 17 21:06:17.970454 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:06:17.970340    2567 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-primary-b61317-predictor-serving-cert: secret "isvc-primary-b61317-predictor-serving-cert" not found
Sep 17 21:06:17.970454 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:06:17.970410    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92420c11-81b3-44aa-ac53-f139e0ae5329-proxy-tls podName:92420c11-81b3-44aa-ac53-f139e0ae5329 nodeName:}" failed. No retries permitted until 2026-09-17 21:06:18.470390367 +0000 UTC m=+1042.815992187 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/92420c11-81b3-44aa-ac53-f139e0ae5329-proxy-tls") pod "isvc-primary-b61317-predictor-6d9bb74dbf-cb8db" (UID: "92420c11-81b3-44aa-ac53-f139e0ae5329") : secret "isvc-primary-b61317-predictor-serving-cert" not found
Sep 17 21:06:17.970721 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.970703    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/92420c11-81b3-44aa-ac53-f139e0ae5329-kserve-provision-location\") pod \"isvc-primary-b61317-predictor-6d9bb74dbf-cb8db\" (UID: \"92420c11-81b3-44aa-ac53-f139e0ae5329\") " pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db"
Sep 17 21:06:17.970936 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.970920    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/92420c11-81b3-44aa-ac53-f139e0ae5329-kube-rbac-proxy-sar-config\") pod \"isvc-primary-b61317-predictor-6d9bb74dbf-cb8db\" (UID: \"92420c11-81b3-44aa-ac53-f139e0ae5329\") " pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db"
Sep 17 21:06:17.970978 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.970951    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/92420c11-81b3-44aa-ac53-f139e0ae5329-cabundle-cert\") pod \"isvc-primary-b61317-predictor-6d9bb74dbf-cb8db\" (UID: \"92420c11-81b3-44aa-ac53-f139e0ae5329\") " pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db"
Sep 17 21:06:17.982441 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:17.982423    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfsdh\" (UniqueName: \"kubernetes.io/projected/92420c11-81b3-44aa-ac53-f139e0ae5329-kube-api-access-bfsdh\") pod \"isvc-primary-b61317-predictor-6d9bb74dbf-cb8db\" (UID: \"92420c11-81b3-44aa-ac53-f139e0ae5329\") " pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db"
Sep 17 21:06:18.049275 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:18.049170    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:06:18.475515 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:18.475461    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/92420c11-81b3-44aa-ac53-f139e0ae5329-proxy-tls\") pod \"isvc-primary-b61317-predictor-6d9bb74dbf-cb8db\" (UID: \"92420c11-81b3-44aa-ac53-f139e0ae5329\") " pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db"
Sep 17 21:06:18.477949 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:18.477916    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/92420c11-81b3-44aa-ac53-f139e0ae5329-proxy-tls\") pod \"isvc-primary-b61317-predictor-6d9bb74dbf-cb8db\" (UID: \"92420c11-81b3-44aa-ac53-f139e0ae5329\") " pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db"
Sep 17 21:06:18.750475 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:18.750376    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db"
Sep 17 21:06:19.024011 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:19.023871    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db"]
Sep 17 21:06:19.026717 ip-10-0-132-201 kubenswrapper[2567]: W0917 21:06:19.026692    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92420c11_81b3_44aa_ac53_f139e0ae5329.slice/crio-9886e2f8ae74953e00f0d50a156e443f31b72a09e1a3c93ffba886fdecd3b4cc WatchSource:0}: Error finding container 9886e2f8ae74953e00f0d50a156e443f31b72a09e1a3c93ffba886fdecd3b4cc: Status 404 returned error can't find the container with id 9886e2f8ae74953e00f0d50a156e443f31b72a09e1a3c93ffba886fdecd3b4cc
Sep 17 21:06:19.333046 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:19.332959    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db" event={"ID":"92420c11-81b3-44aa-ac53-f139e0ae5329","Type":"ContainerStarted","Data":"3b68b738dea36cda0ff5e80ffa839a819cbecaf996d89cfcce7e9a8dc4385c89"}
Sep 17 21:06:19.333046 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:19.333022    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db" event={"ID":"92420c11-81b3-44aa-ac53-f139e0ae5329","Type":"ContainerStarted","Data":"9886e2f8ae74953e00f0d50a156e443f31b72a09e1a3c93ffba886fdecd3b4cc"}
Sep 17 21:06:23.349382 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:23.349350    2567 generic.go:356] "Generic (PLEG): container finished" podID="92420c11-81b3-44aa-ac53-f139e0ae5329" containerID="3b68b738dea36cda0ff5e80ffa839a819cbecaf996d89cfcce7e9a8dc4385c89" exitCode=0
Sep 17 21:06:23.349742 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:23.349402    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db" event={"ID":"92420c11-81b3-44aa-ac53-f139e0ae5329","Type":"ContainerDied","Data":"3b68b738dea36cda0ff5e80ffa839a819cbecaf996d89cfcce7e9a8dc4385c89"}
Sep 17 21:06:24.355277 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:24.355241    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db" event={"ID":"92420c11-81b3-44aa-ac53-f139e0ae5329","Type":"ContainerStarted","Data":"19f0f13f5b17365618a9e89a7953c419f80f0a019be17da2aafed08fd1a27246"}
Sep 17 21:06:24.355700 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:24.355305    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db" event={"ID":"92420c11-81b3-44aa-ac53-f139e0ae5329","Type":"ContainerStarted","Data":"d0116e99bb5125b3080ed5766a51e5603c802010d4243d4d725ef9f5419a6809"}
Sep 17 21:06:24.355700 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:24.355488    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db"
Sep 17 21:06:24.376336 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:24.376286    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db" podStartSLOduration=3.562075982 podStartE2EDuration="7.376271071s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.814195089s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:06:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:06:24.375974574 +0000 UTC m=+1048.721576421" watchObservedRunningTime="2026-09-17 21:06:24.376271071 +0000 UTC m=+1048.721872896"
Sep 17 21:06:25.359852 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:25.359819    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db"
Sep 17 21:06:25.361194 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:25.361165    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db" podUID="92420c11-81b3-44aa-ac53-f139e0ae5329" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused"
Sep 17 21:06:26.363471 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:26.363429    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db" podUID="92420c11-81b3-44aa-ac53-f139e0ae5329" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused"
Sep 17 21:06:31.368481 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:31.368451    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db"
Sep 17 21:06:31.368970 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:31.368942    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db" podUID="92420c11-81b3-44aa-ac53-f139e0ae5329" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused"
Sep 17 21:06:37.393949 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:37.393916    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:06:41.368952 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:41.368911    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db" podUID="92420c11-81b3-44aa-ac53-f139e0ae5329" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused"
Sep 17 21:06:51.369694 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:51.369610    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db" podUID="92420c11-81b3-44aa-ac53-f139e0ae5329" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused"
Sep 17 21:06:53.234444 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:53.234406    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:06:54.498030 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:06:54.497990    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:07:01.369859 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:01.369816    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db" podUID="92420c11-81b3-44aa-ac53-f139e0ae5329" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused"
Sep 17 21:07:03.342323 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:03.342284    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:07:03.482536 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:03.482497    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:07:03.574188 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:03.574143    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:07:03.597605 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:03.597529    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd"]
Sep 17 21:07:03.602233 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:03.602217    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd"
Sep 17 21:07:03.606593 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:03.606570    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"message-dumper-raw-8d8d6-predictor-serving-cert\""
Sep 17 21:07:03.608116 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:03.608085    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"message-dumper-raw-8d8d6-kube-rbac-proxy-sar-config\""
Sep 17 21:07:03.613586 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:03.613564    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd"]
Sep 17 21:07:03.683182 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:03.683140    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:07:03.748615 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:03.748582    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b9a0f56-1418-4247-97b6-fe09195f157f-proxy-tls\") pod \"message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd\" (UID: \"1b9a0f56-1418-4247-97b6-fe09195f157f\") " pod="kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd"
Sep 17 21:07:03.748789 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:03.748651    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1b9a0f56-1418-4247-97b6-fe09195f157f-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd\" (UID: \"1b9a0f56-1418-4247-97b6-fe09195f157f\") " pod="kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd"
Sep 17 21:07:03.748789 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:03.748704    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bm59\" (UniqueName: \"kubernetes.io/projected/1b9a0f56-1418-4247-97b6-fe09195f157f-kube-api-access-7bm59\") pod \"message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd\" (UID: \"1b9a0f56-1418-4247-97b6-fe09195f157f\") " pod="kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd"
Sep 17 21:07:03.849809 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:03.849696    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bm59\" (UniqueName: \"kubernetes.io/projected/1b9a0f56-1418-4247-97b6-fe09195f157f-kube-api-access-7bm59\") pod \"message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd\" (UID: \"1b9a0f56-1418-4247-97b6-fe09195f157f\") " pod="kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd"
Sep 17 21:07:03.849809 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:03.849746    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b9a0f56-1418-4247-97b6-fe09195f157f-proxy-tls\") pod \"message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd\" (UID: \"1b9a0f56-1418-4247-97b6-fe09195f157f\") " pod="kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd"
Sep 17 21:07:03.850048 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:03.849812    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1b9a0f56-1418-4247-97b6-fe09195f157f-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd\" (UID: \"1b9a0f56-1418-4247-97b6-fe09195f157f\") " pod="kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd"
Sep 17 21:07:03.850460 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:03.850432    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1b9a0f56-1418-4247-97b6-fe09195f157f-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd\" (UID: \"1b9a0f56-1418-4247-97b6-fe09195f157f\") " pod="kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd"
Sep 17 21:07:03.852186 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:03.852164    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b9a0f56-1418-4247-97b6-fe09195f157f-proxy-tls\") pod \"message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd\" (UID: \"1b9a0f56-1418-4247-97b6-fe09195f157f\") " pod="kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd"
Sep 17 21:07:03.870383 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:03.870353    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bm59\" (UniqueName: \"kubernetes.io/projected/1b9a0f56-1418-4247-97b6-fe09195f157f-kube-api-access-7bm59\") pod \"message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd\" (UID: \"1b9a0f56-1418-4247-97b6-fe09195f157f\") " pod="kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd"
Sep 17 21:07:03.913135 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:03.913103    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd"
Sep 17 21:07:03.922557 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:03.922526    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:07:04.190431 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:04.190408    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd"]
Sep 17 21:07:04.192307 ip-10-0-132-201 kubenswrapper[2567]: W0917 21:07:04.192284    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b9a0f56_1418_4247_97b6_fe09195f157f.slice/crio-d4452e2974a5a6bdc69db4525f5ecf39b56e5fe7df6c2fdbbeaec6e83d53dc3e WatchSource:0}: Error finding container d4452e2974a5a6bdc69db4525f5ecf39b56e5fe7df6c2fdbbeaec6e83d53dc3e: Status 404 returned error can't find the container with id d4452e2974a5a6bdc69db4525f5ecf39b56e5fe7df6c2fdbbeaec6e83d53dc3e
Sep 17 21:07:04.520509 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:04.520474    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd" event={"ID":"1b9a0f56-1418-4247-97b6-fe09195f157f","Type":"ContainerStarted","Data":"d4452e2974a5a6bdc69db4525f5ecf39b56e5fe7df6c2fdbbeaec6e83d53dc3e"}
Sep 17 21:07:05.526176 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:05.526132    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd" event={"ID":"1b9a0f56-1418-4247-97b6-fe09195f157f","Type":"ContainerStarted","Data":"966f28ab66d185aae45123a82a9db896d88696460af2a817fce2941074de3650"}
Sep 17 21:07:05.526607 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:05.526198    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd" event={"ID":"1b9a0f56-1418-4247-97b6-fe09195f157f","Type":"ContainerStarted","Data":"2a51a81be47f400e2eaedd46f3310fec0703c21323b56568a4e5724f847b2de7"}
Sep 17 21:07:05.526607 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:05.526327    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd"
Sep 17 21:07:05.526607 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:05.526357    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd"
Sep 17 21:07:05.528112 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:05.528090    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd"
Sep 17 21:07:05.549111 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:05.549066    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd" podStartSLOduration=1.404053218 podStartE2EDuration="2.549052349s" totalImagesPullingTime="1.144999131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:07:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:07:05.547604324 +0000 UTC m=+1089.893206171" watchObservedRunningTime="2026-09-17 21:07:05.549052349 +0000 UTC m=+1089.894654176"
Sep 17 21:07:11.369956 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:11.369919    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db" podUID="92420c11-81b3-44aa-ac53-f139e0ae5329" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused"
Sep 17 21:07:12.539597 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:12.539566    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd"
Sep 17 21:07:13.494195 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:13.494154    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:07:13.894325 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:13.894240    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:07:21.368939 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:21.368897    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db" podUID="92420c11-81b3-44aa-ac53-f139e0ae5329" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused"
Sep 17 21:07:31.370280 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:31.370234    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db" podUID="92420c11-81b3-44aa-ac53-f139e0ae5329" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused"
Sep 17 21:07:41.369661 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:07:41.369629    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db"
Sep 17 21:08:02.959403 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:02.959369    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:08:03.272593 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:03.272509    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:08:03.571980 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:03.571889    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:08:19.164466 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:19.164389    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:08:19.171446 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:19.171421    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db"]
Sep 17 21:08:19.171662 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:19.171636    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db" podUID="92420c11-81b3-44aa-ac53-f139e0ae5329" containerName="kserve-container" containerID="cri-o://d0116e99bb5125b3080ed5766a51e5603c802010d4243d4d725ef9f5419a6809" gracePeriod=30
Sep 17 21:08:19.171706 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:19.171662    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db" podUID="92420c11-81b3-44aa-ac53-f139e0ae5329" containerName="kube-rbac-proxy" containerID="cri-o://19f0f13f5b17365618a9e89a7953c419f80f0a019be17da2aafed08fd1a27246" gracePeriod=30
Sep 17 21:08:19.224624 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:19.224593    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:08:19.827660 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:19.827624    2567 generic.go:356] "Generic (PLEG): container finished" podID="92420c11-81b3-44aa-ac53-f139e0ae5329" containerID="19f0f13f5b17365618a9e89a7953c419f80f0a019be17da2aafed08fd1a27246" exitCode=2
Sep 17 21:08:19.827871 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:19.827700    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db" event={"ID":"92420c11-81b3-44aa-ac53-f139e0ae5329","Type":"ContainerDied","Data":"19f0f13f5b17365618a9e89a7953c419f80f0a019be17da2aafed08fd1a27246"}
Sep 17 21:08:21.364202 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:21.364151    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db" podUID="92420c11-81b3-44aa-ac53-f139e0ae5329" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.29:8643/healthz\": dial tcp 10.132.0.29:8643: connect: connection refused"
Sep 17 21:08:21.369684 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:21.369650    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db" podUID="92420c11-81b3-44aa-ac53-f139e0ae5329" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused"
Sep 17 21:08:24.309016 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.308927    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:08:24.418429 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.418390    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:08:24.535296 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.534972    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db"
Sep 17 21:08:24.649462 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.649368    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/92420c11-81b3-44aa-ac53-f139e0ae5329-proxy-tls\") pod \"92420c11-81b3-44aa-ac53-f139e0ae5329\" (UID: \"92420c11-81b3-44aa-ac53-f139e0ae5329\") "
Sep 17 21:08:24.649721 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.649699    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfsdh\" (UniqueName: \"kubernetes.io/projected/92420c11-81b3-44aa-ac53-f139e0ae5329-kube-api-access-bfsdh\") pod \"92420c11-81b3-44aa-ac53-f139e0ae5329\" (UID: \"92420c11-81b3-44aa-ac53-f139e0ae5329\") "
Sep 17 21:08:24.649929 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.649911    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/92420c11-81b3-44aa-ac53-f139e0ae5329-cabundle-cert\") pod \"92420c11-81b3-44aa-ac53-f139e0ae5329\" (UID: \"92420c11-81b3-44aa-ac53-f139e0ae5329\") "
Sep 17 21:08:24.650051 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.650036    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/92420c11-81b3-44aa-ac53-f139e0ae5329-kserve-provision-location\") pod \"92420c11-81b3-44aa-ac53-f139e0ae5329\" (UID: \"92420c11-81b3-44aa-ac53-f139e0ae5329\") "
Sep 17 21:08:24.650204 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.650189    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/92420c11-81b3-44aa-ac53-f139e0ae5329-kube-rbac-proxy-sar-config\") pod \"92420c11-81b3-44aa-ac53-f139e0ae5329\" (UID: \"92420c11-81b3-44aa-ac53-f139e0ae5329\") "
Sep 17 21:08:24.650599 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.650564    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92420c11-81b3-44aa-ac53-f139e0ae5329-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "92420c11-81b3-44aa-ac53-f139e0ae5329" (UID: "92420c11-81b3-44aa-ac53-f139e0ae5329"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:08:24.650836 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.650809    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92420c11-81b3-44aa-ac53-f139e0ae5329-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "92420c11-81b3-44aa-ac53-f139e0ae5329" (UID: "92420c11-81b3-44aa-ac53-f139e0ae5329"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 21:08:24.651100 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.651077    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92420c11-81b3-44aa-ac53-f139e0ae5329-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "92420c11-81b3-44aa-ac53-f139e0ae5329" (UID: "92420c11-81b3-44aa-ac53-f139e0ae5329"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:08:24.659776 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.653913    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92420c11-81b3-44aa-ac53-f139e0ae5329-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "92420c11-81b3-44aa-ac53-f139e0ae5329" (UID: "92420c11-81b3-44aa-ac53-f139e0ae5329"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:08:24.665242 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.665219    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92420c11-81b3-44aa-ac53-f139e0ae5329-kube-api-access-bfsdh" (OuterVolumeSpecName: "kube-api-access-bfsdh") pod "92420c11-81b3-44aa-ac53-f139e0ae5329" (UID: "92420c11-81b3-44aa-ac53-f139e0ae5329"). InnerVolumeSpecName "kube-api-access-bfsdh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:08:24.751704 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.751672    2567 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/92420c11-81b3-44aa-ac53-f139e0ae5329-cabundle-cert\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:08:24.751704 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.751702    2567 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/92420c11-81b3-44aa-ac53-f139e0ae5329-kserve-provision-location\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:08:24.751961 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.751716    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/92420c11-81b3-44aa-ac53-f139e0ae5329-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:08:24.751961 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.751730    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/92420c11-81b3-44aa-ac53-f139e0ae5329-proxy-tls\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:08:24.751961 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.751743    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfsdh\" (UniqueName: \"kubernetes.io/projected/92420c11-81b3-44aa-ac53-f139e0ae5329-kube-api-access-bfsdh\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:08:24.862296 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.862165    2567 generic.go:356] "Generic (PLEG): container finished" podID="92420c11-81b3-44aa-ac53-f139e0ae5329" containerID="d0116e99bb5125b3080ed5766a51e5603c802010d4243d4d725ef9f5419a6809" exitCode=0
Sep 17 21:08:24.862595 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.862259    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db" event={"ID":"92420c11-81b3-44aa-ac53-f139e0ae5329","Type":"ContainerDied","Data":"d0116e99bb5125b3080ed5766a51e5603c802010d4243d4d725ef9f5419a6809"}
Sep 17 21:08:24.862778 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.862725    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db" event={"ID":"92420c11-81b3-44aa-ac53-f139e0ae5329","Type":"ContainerDied","Data":"9886e2f8ae74953e00f0d50a156e443f31b72a09e1a3c93ffba886fdecd3b4cc"}
Sep 17 21:08:24.862972 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.862931    2567 scope.go:117] "RemoveContainer" containerID="19f0f13f5b17365618a9e89a7953c419f80f0a019be17da2aafed08fd1a27246"
Sep 17 21:08:24.863397 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.863291    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db"
Sep 17 21:08:24.887198 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.887174    2567 scope.go:117] "RemoveContainer" containerID="d0116e99bb5125b3080ed5766a51e5603c802010d4243d4d725ef9f5419a6809"
Sep 17 21:08:24.894957 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.894929    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db"]
Sep 17 21:08:24.896125 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.896107    2567 scope.go:117] "RemoveContainer" containerID="3b68b738dea36cda0ff5e80ffa839a819cbecaf996d89cfcce7e9a8dc4385c89"
Sep 17 21:08:24.897503 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.897476    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-primary-b61317-predictor-6d9bb74dbf-cb8db"]
Sep 17 21:08:24.903709 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.903688    2567 scope.go:117] "RemoveContainer" containerID="19f0f13f5b17365618a9e89a7953c419f80f0a019be17da2aafed08fd1a27246"
Sep 17 21:08:24.903993 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:08:24.903975    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19f0f13f5b17365618a9e89a7953c419f80f0a019be17da2aafed08fd1a27246\": container with ID starting with 19f0f13f5b17365618a9e89a7953c419f80f0a019be17da2aafed08fd1a27246 not found: ID does not exist" containerID="19f0f13f5b17365618a9e89a7953c419f80f0a019be17da2aafed08fd1a27246"
Sep 17 21:08:24.904049 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.904005    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19f0f13f5b17365618a9e89a7953c419f80f0a019be17da2aafed08fd1a27246"} err="failed to get container status \"19f0f13f5b17365618a9e89a7953c419f80f0a019be17da2aafed08fd1a27246\": rpc error: code = NotFound desc = could not find container \"19f0f13f5b17365618a9e89a7953c419f80f0a019be17da2aafed08fd1a27246\": container with ID starting with 19f0f13f5b17365618a9e89a7953c419f80f0a019be17da2aafed08fd1a27246 not found: ID does not exist"
Sep 17 21:08:24.904049 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.904023    2567 scope.go:117] "RemoveContainer" containerID="d0116e99bb5125b3080ed5766a51e5603c802010d4243d4d725ef9f5419a6809"
Sep 17 21:08:24.904294 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:08:24.904276    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0116e99bb5125b3080ed5766a51e5603c802010d4243d4d725ef9f5419a6809\": container with ID starting with d0116e99bb5125b3080ed5766a51e5603c802010d4243d4d725ef9f5419a6809 not found: ID does not exist" containerID="d0116e99bb5125b3080ed5766a51e5603c802010d4243d4d725ef9f5419a6809"
Sep 17 21:08:24.904347 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.904300    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0116e99bb5125b3080ed5766a51e5603c802010d4243d4d725ef9f5419a6809"} err="failed to get container status \"d0116e99bb5125b3080ed5766a51e5603c802010d4243d4d725ef9f5419a6809\": rpc error: code = NotFound desc = could not find container \"d0116e99bb5125b3080ed5766a51e5603c802010d4243d4d725ef9f5419a6809\": container with ID starting with d0116e99bb5125b3080ed5766a51e5603c802010d4243d4d725ef9f5419a6809 not found: ID does not exist"
Sep 17 21:08:24.904347 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.904314    2567 scope.go:117] "RemoveContainer" containerID="3b68b738dea36cda0ff5e80ffa839a819cbecaf996d89cfcce7e9a8dc4385c89"
Sep 17 21:08:24.904535 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:08:24.904518    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b68b738dea36cda0ff5e80ffa839a819cbecaf996d89cfcce7e9a8dc4385c89\": container with ID starting with 3b68b738dea36cda0ff5e80ffa839a819cbecaf996d89cfcce7e9a8dc4385c89 not found: ID does not exist" containerID="3b68b738dea36cda0ff5e80ffa839a819cbecaf996d89cfcce7e9a8dc4385c89"
Sep 17 21:08:24.904573 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:24.904541    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b68b738dea36cda0ff5e80ffa839a819cbecaf996d89cfcce7e9a8dc4385c89"} err="failed to get container status \"3b68b738dea36cda0ff5e80ffa839a819cbecaf996d89cfcce7e9a8dc4385c89\": rpc error: code = NotFound desc = could not find container \"3b68b738dea36cda0ff5e80ffa839a819cbecaf996d89cfcce7e9a8dc4385c89\": container with ID starting with 3b68b738dea36cda0ff5e80ffa839a819cbecaf996d89cfcce7e9a8dc4385c89 not found: ID does not exist"
Sep 17 21:08:26.170225 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:26.170193    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92420c11-81b3-44aa-ac53-f139e0ae5329" path="/var/lib/kubelet/pods/92420c11-81b3-44aa-ac53-f139e0ae5329/volumes"
Sep 17 21:08:29.412858 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:29.412825    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:08:34.367422 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:34.367386    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:08:39.545036 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:39.545002    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:08:39.646588 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:39.646544    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:08:49.026854 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.026814    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd_1b9a0f56-1418-4247-97b6-fe09195f157f/kserve-container/0.log"
Sep 17 21:08:49.201098 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.201060    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:08:49.201972 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.201945    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd"]
Sep 17 21:08:49.202240 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.202190    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd" podUID="1b9a0f56-1418-4247-97b6-fe09195f157f" containerName="kserve-container" containerID="cri-o://2a51a81be47f400e2eaedd46f3310fec0703c21323b56568a4e5724f847b2de7" gracePeriod=30
Sep 17 21:08:49.202351 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.202232    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd" podUID="1b9a0f56-1418-4247-97b6-fe09195f157f" containerName="kube-rbac-proxy" containerID="cri-o://966f28ab66d185aae45123a82a9db896d88696460af2a817fce2941074de3650" gracePeriod=30
Sep 17 21:08:49.271411 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.271377    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:08:49.288336 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.288268    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"]
Sep 17 21:08:49.288596 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.288584    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92420c11-81b3-44aa-ac53-f139e0ae5329" containerName="storage-initializer"
Sep 17 21:08:49.288647 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.288598    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="92420c11-81b3-44aa-ac53-f139e0ae5329" containerName="storage-initializer"
Sep 17 21:08:49.288647 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.288612    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92420c11-81b3-44aa-ac53-f139e0ae5329" containerName="kserve-container"
Sep 17 21:08:49.288647 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.288617    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="92420c11-81b3-44aa-ac53-f139e0ae5329" containerName="kserve-container"
Sep 17 21:08:49.288748 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.288628    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92420c11-81b3-44aa-ac53-f139e0ae5329" containerName="kube-rbac-proxy"
Sep 17 21:08:49.288748 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.288667    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="92420c11-81b3-44aa-ac53-f139e0ae5329" containerName="kube-rbac-proxy"
Sep 17 21:08:49.288748 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.288740    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="92420c11-81b3-44aa-ac53-f139e0ae5329" containerName="kube-rbac-proxy"
Sep 17 21:08:49.288868 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.288750    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="92420c11-81b3-44aa-ac53-f139e0ae5329" containerName="kserve-container"
Sep 17 21:08:49.292286 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.292232    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"
Sep 17 21:08:49.294975 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.294952    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-scale-raw-f9ecc-kube-rbac-proxy-sar-config\""
Sep 17 21:08:49.295092 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.294952    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Sep 17 21:08:49.295092 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.294985    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Sep 17 21:08:49.295092 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.294956    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-scale-raw-f9ecc-predictor-serving-cert\""
Sep 17 21:08:49.303101 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.303079    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"]
Sep 17 21:08:49.347057 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.347018    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sdth\" (UniqueName: \"kubernetes.io/projected/3ad76ac7-1b45-4a8b-9472-a3902021e09f-kube-api-access-6sdth\") pod \"isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7\" (UID: \"3ad76ac7-1b45-4a8b-9472-a3902021e09f\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"
Sep 17 21:08:49.347057 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.347066    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3ad76ac7-1b45-4a8b-9472-a3902021e09f-kserve-provision-location\") pod \"isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7\" (UID: \"3ad76ac7-1b45-4a8b-9472-a3902021e09f\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"
Sep 17 21:08:49.347315 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.347145    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3ad76ac7-1b45-4a8b-9472-a3902021e09f-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7\" (UID: \"3ad76ac7-1b45-4a8b-9472-a3902021e09f\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"
Sep 17 21:08:49.347315 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.347218    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3ad76ac7-1b45-4a8b-9472-a3902021e09f-cabundle-cert\") pod \"isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7\" (UID: \"3ad76ac7-1b45-4a8b-9472-a3902021e09f\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"
Sep 17 21:08:49.347315 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.347251    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3ad76ac7-1b45-4a8b-9472-a3902021e09f-proxy-tls\") pod \"isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7\" (UID: \"3ad76ac7-1b45-4a8b-9472-a3902021e09f\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"
Sep 17 21:08:49.447575 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.447544    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3ad76ac7-1b45-4a8b-9472-a3902021e09f-proxy-tls\") pod \"isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7\" (UID: \"3ad76ac7-1b45-4a8b-9472-a3902021e09f\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"
Sep 17 21:08:49.447725 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.447587    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sdth\" (UniqueName: \"kubernetes.io/projected/3ad76ac7-1b45-4a8b-9472-a3902021e09f-kube-api-access-6sdth\") pod \"isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7\" (UID: \"3ad76ac7-1b45-4a8b-9472-a3902021e09f\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"
Sep 17 21:08:49.447725 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.447605    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3ad76ac7-1b45-4a8b-9472-a3902021e09f-kserve-provision-location\") pod \"isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7\" (UID: \"3ad76ac7-1b45-4a8b-9472-a3902021e09f\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"
Sep 17 21:08:49.447725 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.447660    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3ad76ac7-1b45-4a8b-9472-a3902021e09f-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7\" (UID: \"3ad76ac7-1b45-4a8b-9472-a3902021e09f\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"
Sep 17 21:08:49.447725 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.447704    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3ad76ac7-1b45-4a8b-9472-a3902021e09f-cabundle-cert\") pod \"isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7\" (UID: \"3ad76ac7-1b45-4a8b-9472-a3902021e09f\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"
Sep 17 21:08:49.447725 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:08:49.447717    2567 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-serving-cert: secret "isvc-sklearn-scale-raw-f9ecc-predictor-serving-cert" not found
Sep 17 21:08:49.448053 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:08:49.447809    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ad76ac7-1b45-4a8b-9472-a3902021e09f-proxy-tls podName:3ad76ac7-1b45-4a8b-9472-a3902021e09f nodeName:}" failed. No retries permitted until 2026-09-17 21:08:49.947788688 +0000 UTC m=+1194.293390509 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/3ad76ac7-1b45-4a8b-9472-a3902021e09f-proxy-tls") pod "isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" (UID: "3ad76ac7-1b45-4a8b-9472-a3902021e09f") : secret "isvc-sklearn-scale-raw-f9ecc-predictor-serving-cert" not found
Sep 17 21:08:49.448230 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.448204    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3ad76ac7-1b45-4a8b-9472-a3902021e09f-kserve-provision-location\") pod \"isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7\" (UID: \"3ad76ac7-1b45-4a8b-9472-a3902021e09f\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"
Sep 17 21:08:49.448408 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.448391    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3ad76ac7-1b45-4a8b-9472-a3902021e09f-cabundle-cert\") pod \"isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7\" (UID: \"3ad76ac7-1b45-4a8b-9472-a3902021e09f\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"
Sep 17 21:08:49.448539 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.448519    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3ad76ac7-1b45-4a8b-9472-a3902021e09f-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7\" (UID: \"3ad76ac7-1b45-4a8b-9472-a3902021e09f\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"
Sep 17 21:08:49.456428 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.456404    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sdth\" (UniqueName: \"kubernetes.io/projected/3ad76ac7-1b45-4a8b-9472-a3902021e09f-kube-api-access-6sdth\") pod \"isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7\" (UID: \"3ad76ac7-1b45-4a8b-9472-a3902021e09f\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"
Sep 17 21:08:49.460697 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.460677    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd"
Sep 17 21:08:49.548140 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.548043    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b9a0f56-1418-4247-97b6-fe09195f157f-proxy-tls\") pod \"1b9a0f56-1418-4247-97b6-fe09195f157f\" (UID: \"1b9a0f56-1418-4247-97b6-fe09195f157f\") "
Sep 17 21:08:49.548140 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.548132    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7bm59\" (UniqueName: \"kubernetes.io/projected/1b9a0f56-1418-4247-97b6-fe09195f157f-kube-api-access-7bm59\") pod \"1b9a0f56-1418-4247-97b6-fe09195f157f\" (UID: \"1b9a0f56-1418-4247-97b6-fe09195f157f\") "
Sep 17 21:08:49.548374 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.548177    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1b9a0f56-1418-4247-97b6-fe09195f157f-kube-rbac-proxy-sar-config\") pod \"1b9a0f56-1418-4247-97b6-fe09195f157f\" (UID: \"1b9a0f56-1418-4247-97b6-fe09195f157f\") "
Sep 17 21:08:49.548580 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.548555    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b9a0f56-1418-4247-97b6-fe09195f157f-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "1b9a0f56-1418-4247-97b6-fe09195f157f" (UID: "1b9a0f56-1418-4247-97b6-fe09195f157f"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:08:49.550267 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.550238    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b9a0f56-1418-4247-97b6-fe09195f157f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "1b9a0f56-1418-4247-97b6-fe09195f157f" (UID: "1b9a0f56-1418-4247-97b6-fe09195f157f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:08:49.550369 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.550279    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b9a0f56-1418-4247-97b6-fe09195f157f-kube-api-access-7bm59" (OuterVolumeSpecName: "kube-api-access-7bm59") pod "1b9a0f56-1418-4247-97b6-fe09195f157f" (UID: "1b9a0f56-1418-4247-97b6-fe09195f157f"). InnerVolumeSpecName "kube-api-access-7bm59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:08:49.621651 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.621622    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:08:49.649306 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.649285    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1b9a0f56-1418-4247-97b6-fe09195f157f-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:08:49.649306 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.649306    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b9a0f56-1418-4247-97b6-fe09195f157f-proxy-tls\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:08:49.649459 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.649315    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bm59\" (UniqueName: \"kubernetes.io/projected/1b9a0f56-1418-4247-97b6-fe09195f157f-kube-api-access-7bm59\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:08:49.952576 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.952480    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3ad76ac7-1b45-4a8b-9472-a3902021e09f-proxy-tls\") pod \"isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7\" (UID: \"3ad76ac7-1b45-4a8b-9472-a3902021e09f\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"
Sep 17 21:08:49.954895 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.954874    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3ad76ac7-1b45-4a8b-9472-a3902021e09f-proxy-tls\") pod \"isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7\" (UID: \"3ad76ac7-1b45-4a8b-9472-a3902021e09f\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"
Sep 17 21:08:49.963546 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.963519    2567 generic.go:356] "Generic (PLEG): container finished" podID="1b9a0f56-1418-4247-97b6-fe09195f157f" containerID="966f28ab66d185aae45123a82a9db896d88696460af2a817fce2941074de3650" exitCode=2
Sep 17 21:08:49.963546 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.963543    2567 generic.go:356] "Generic (PLEG): container finished" podID="1b9a0f56-1418-4247-97b6-fe09195f157f" containerID="2a51a81be47f400e2eaedd46f3310fec0703c21323b56568a4e5724f847b2de7" exitCode=2
Sep 17 21:08:49.963696 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.963563    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd"
Sep 17 21:08:49.963696 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.963600    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd" event={"ID":"1b9a0f56-1418-4247-97b6-fe09195f157f","Type":"ContainerDied","Data":"966f28ab66d185aae45123a82a9db896d88696460af2a817fce2941074de3650"}
Sep 17 21:08:49.963696 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.963682    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd" event={"ID":"1b9a0f56-1418-4247-97b6-fe09195f157f","Type":"ContainerDied","Data":"2a51a81be47f400e2eaedd46f3310fec0703c21323b56568a4e5724f847b2de7"}
Sep 17 21:08:49.963854 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.963708    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd" event={"ID":"1b9a0f56-1418-4247-97b6-fe09195f157f","Type":"ContainerDied","Data":"d4452e2974a5a6bdc69db4525f5ecf39b56e5fe7df6c2fdbbeaec6e83d53dc3e"}
Sep 17 21:08:49.963854 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.963738    2567 scope.go:117] "RemoveContainer" containerID="966f28ab66d185aae45123a82a9db896d88696460af2a817fce2941074de3650"
Sep 17 21:08:49.972303 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.972283    2567 scope.go:117] "RemoveContainer" containerID="2a51a81be47f400e2eaedd46f3310fec0703c21323b56568a4e5724f847b2de7"
Sep 17 21:08:49.979610 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.979595    2567 scope.go:117] "RemoveContainer" containerID="966f28ab66d185aae45123a82a9db896d88696460af2a817fce2941074de3650"
Sep 17 21:08:49.979869 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:08:49.979850    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"966f28ab66d185aae45123a82a9db896d88696460af2a817fce2941074de3650\": container with ID starting with 966f28ab66d185aae45123a82a9db896d88696460af2a817fce2941074de3650 not found: ID does not exist" containerID="966f28ab66d185aae45123a82a9db896d88696460af2a817fce2941074de3650"
Sep 17 21:08:49.979936 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.979875    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"966f28ab66d185aae45123a82a9db896d88696460af2a817fce2941074de3650"} err="failed to get container status \"966f28ab66d185aae45123a82a9db896d88696460af2a817fce2941074de3650\": rpc error: code = NotFound desc = could not find container \"966f28ab66d185aae45123a82a9db896d88696460af2a817fce2941074de3650\": container with ID starting with 966f28ab66d185aae45123a82a9db896d88696460af2a817fce2941074de3650 not found: ID does not exist"
Sep 17 21:08:49.979936 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.979890    2567 scope.go:117] "RemoveContainer" containerID="2a51a81be47f400e2eaedd46f3310fec0703c21323b56568a4e5724f847b2de7"
Sep 17 21:08:49.980122 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:08:49.980103    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a51a81be47f400e2eaedd46f3310fec0703c21323b56568a4e5724f847b2de7\": container with ID starting with 2a51a81be47f400e2eaedd46f3310fec0703c21323b56568a4e5724f847b2de7 not found: ID does not exist" containerID="2a51a81be47f400e2eaedd46f3310fec0703c21323b56568a4e5724f847b2de7"
Sep 17 21:08:49.980166 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.980128    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a51a81be47f400e2eaedd46f3310fec0703c21323b56568a4e5724f847b2de7"} err="failed to get container status \"2a51a81be47f400e2eaedd46f3310fec0703c21323b56568a4e5724f847b2de7\": rpc error: code = NotFound desc = could not find container \"2a51a81be47f400e2eaedd46f3310fec0703c21323b56568a4e5724f847b2de7\": container with ID starting with 2a51a81be47f400e2eaedd46f3310fec0703c21323b56568a4e5724f847b2de7 not found: ID does not exist"
Sep 17 21:08:49.980166 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.980145    2567 scope.go:117] "RemoveContainer" containerID="966f28ab66d185aae45123a82a9db896d88696460af2a817fce2941074de3650"
Sep 17 21:08:49.980372 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.980353    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"966f28ab66d185aae45123a82a9db896d88696460af2a817fce2941074de3650"} err="failed to get container status \"966f28ab66d185aae45123a82a9db896d88696460af2a817fce2941074de3650\": rpc error: code = NotFound desc = could not find container \"966f28ab66d185aae45123a82a9db896d88696460af2a817fce2941074de3650\": container with ID starting with 966f28ab66d185aae45123a82a9db896d88696460af2a817fce2941074de3650 not found: ID does not exist"
Sep 17 21:08:49.980429 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.980372    2567 scope.go:117] "RemoveContainer" containerID="2a51a81be47f400e2eaedd46f3310fec0703c21323b56568a4e5724f847b2de7"
Sep 17 21:08:49.980558 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.980542    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a51a81be47f400e2eaedd46f3310fec0703c21323b56568a4e5724f847b2de7"} err="failed to get container status \"2a51a81be47f400e2eaedd46f3310fec0703c21323b56568a4e5724f847b2de7\": rpc error: code = NotFound desc = could not find container \"2a51a81be47f400e2eaedd46f3310fec0703c21323b56568a4e5724f847b2de7\": container with ID starting with 2a51a81be47f400e2eaedd46f3310fec0703c21323b56568a4e5724f847b2de7 not found: ID does not exist"
Sep 17 21:08:49.987847 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.987822    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd"]
Sep 17 21:08:49.991262 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:49.991241    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-8d8d6-predictor-7f84fb8b74-tt7nd"]
Sep 17 21:08:50.169184 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:50.169150    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b9a0f56-1418-4247-97b6-fe09195f157f" path="/var/lib/kubelet/pods/1b9a0f56-1418-4247-97b6-fe09195f157f/volumes"
Sep 17 21:08:50.202826 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:50.202770    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"
Sep 17 21:08:50.476735 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:50.476576    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"]
Sep 17 21:08:50.479706 ip-10-0-132-201 kubenswrapper[2567]: W0917 21:08:50.479668    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ad76ac7_1b45_4a8b_9472_a3902021e09f.slice/crio-985acdfb6dbb9bbe24b2ecdf96e3016c9fb7fe3e9c843f8c028994754397cb0a WatchSource:0}: Error finding container 985acdfb6dbb9bbe24b2ecdf96e3016c9fb7fe3e9c843f8c028994754397cb0a: Status 404 returned error can't find the container with id 985acdfb6dbb9bbe24b2ecdf96e3016c9fb7fe3e9c843f8c028994754397cb0a
Sep 17 21:08:50.970036 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:50.969997    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" event={"ID":"3ad76ac7-1b45-4a8b-9472-a3902021e09f","Type":"ContainerStarted","Data":"88ec544f3ca4ead1615a823f9a0ead28116b16b8c5c9ee412139f9f272047b10"}
Sep 17 21:08:50.970241 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:50.970054    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" event={"ID":"3ad76ac7-1b45-4a8b-9472-a3902021e09f","Type":"ContainerStarted","Data":"985acdfb6dbb9bbe24b2ecdf96e3016c9fb7fe3e9c843f8c028994754397cb0a"}
Sep 17 21:08:54.986155 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:54.986120    2567 generic.go:356] "Generic (PLEG): container finished" podID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerID="88ec544f3ca4ead1615a823f9a0ead28116b16b8c5c9ee412139f9f272047b10" exitCode=0
Sep 17 21:08:54.986573 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:54.986209    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" event={"ID":"3ad76ac7-1b45-4a8b-9472-a3902021e09f","Type":"ContainerDied","Data":"88ec544f3ca4ead1615a823f9a0ead28116b16b8c5c9ee412139f9f272047b10"}
Sep 17 21:08:55.993033 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:55.992996    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" event={"ID":"3ad76ac7-1b45-4a8b-9472-a3902021e09f","Type":"ContainerStarted","Data":"077b77af4efa0f71394430fa9b88839bb9071837eff118b9bb5045bb00f80d2f"}
Sep 17 21:08:55.993463 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:55.993059    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" event={"ID":"3ad76ac7-1b45-4a8b-9472-a3902021e09f","Type":"ContainerStarted","Data":"1a248d32bb0b0575b14af02bf6ffd033872a834ed8a2306d76068f38d4f0a7bf"}
Sep 17 21:08:55.993463 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:55.993315    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"
Sep 17 21:08:55.993463 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:55.993455    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"
Sep 17 21:08:55.994884 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:55.994857    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 17 21:08:56.015040 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:56.014999    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" podStartSLOduration=3.217952934 podStartE2EDuration="7.014986152s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.797033218s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:08:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:08:56.012907267 +0000 UTC m=+1200.358509134" watchObservedRunningTime="2026-09-17 21:08:56.014986152 +0000 UTC m=+1200.360587978"
Sep 17 21:08:56.996852 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:08:56.996809    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 17 21:09:02.001317 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:09:02.001285    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"
Sep 17 21:09:02.002026 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:09:02.001979    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 17 21:09:12.002636 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:09:12.002592    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 17 21:09:22.002438 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:09:22.002399    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 17 21:09:32.002890 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:09:32.002833    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 17 21:09:40.286001 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:09:40.285960    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:09:40.343102 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:09:40.343068    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:09:42.002625 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:09:42.002578    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 17 21:09:52.002583 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:09:52.002488    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 17 21:10:02.002878 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:10:02.002830    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 17 21:10:10.119676 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:10:10.119637    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:10:12.002773 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:10:12.002719    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 17 21:10:13.165165 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:10:13.165126    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 17 21:10:15.259048 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:10:15.259011    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:10:15.898050 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:10:15.898016    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:10:23.165563 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:10:23.165520    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 17 21:10:33.165276 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:10:33.165231    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 17 21:10:43.165888 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:10:43.165843    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 17 21:10:53.165323 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:10:53.165271    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 17 21:11:03.165464 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:03.165421    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 17 21:11:13.165059 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:13.165015    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 17 21:11:16.166387 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:16.166347    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 17 21:11:26.167105 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:26.167066    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 17 21:11:36.169239 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:36.169207    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"
Sep 17 21:11:40.448622 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:40.448590    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"]
Sep 17 21:11:40.449036 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:40.448819    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kserve-container" containerID="cri-o://1a248d32bb0b0575b14af02bf6ffd033872a834ed8a2306d76068f38d4f0a7bf" gracePeriod=30
Sep 17 21:11:40.449036 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:40.448878    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kube-rbac-proxy" containerID="cri-o://077b77af4efa0f71394430fa9b88839bb9071837eff118b9bb5045bb00f80d2f" gracePeriod=30
Sep 17 21:11:40.475195 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:40.475163    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:11:40.647585 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:40.647555    2567 generic.go:356] "Generic (PLEG): container finished" podID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerID="077b77af4efa0f71394430fa9b88839bb9071837eff118b9bb5045bb00f80d2f" exitCode=2
Sep 17 21:11:40.647728 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:40.647606    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" event={"ID":"3ad76ac7-1b45-4a8b-9472-a3902021e09f","Type":"ContainerDied","Data":"077b77af4efa0f71394430fa9b88839bb9071837eff118b9bb5045bb00f80d2f"}
Sep 17 21:11:41.997474 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:41.997435    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.31:8643/healthz\": dial tcp 10.132.0.31:8643: connect: connection refused"
Sep 17 21:11:45.653263 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.653229    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:11:45.716988 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.716960    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf"]
Sep 17 21:11:45.717327 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.717315    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b9a0f56-1418-4247-97b6-fe09195f157f" containerName="kube-rbac-proxy"
Sep 17 21:11:45.717377 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.717330    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b9a0f56-1418-4247-97b6-fe09195f157f" containerName="kube-rbac-proxy"
Sep 17 21:11:45.717377 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.717339    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b9a0f56-1418-4247-97b6-fe09195f157f" containerName="kserve-container"
Sep 17 21:11:45.717377 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.717345    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b9a0f56-1418-4247-97b6-fe09195f157f" containerName="kserve-container"
Sep 17 21:11:45.717469 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.717424    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b9a0f56-1418-4247-97b6-fe09195f157f" containerName="kube-rbac-proxy"
Sep 17 21:11:45.717469 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.717435    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b9a0f56-1418-4247-97b6-fe09195f157f" containerName="kserve-container"
Sep 17 21:11:45.720643 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.720622    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf"
Sep 17 21:11:45.723831 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.723799    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-create-v2-predictor-serving-cert\""
Sep 17 21:11:45.728902 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.728885    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-create-kube-rbac-proxy-sar-config\""
Sep 17 21:11:45.756081 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.756054    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf"]
Sep 17 21:11:45.830930 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.830898    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e72d4ce2-ee89-4167-9628-05705846e4ca-proxy-tls\") pod \"isvc-canary-create-v2-predictor-bfcdb48dd-jggsf\" (UID: \"e72d4ce2-ee89-4167-9628-05705846e4ca\") " pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf"
Sep 17 21:11:45.831099 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.830938    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e72d4ce2-ee89-4167-9628-05705846e4ca-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-v2-predictor-bfcdb48dd-jggsf\" (UID: \"e72d4ce2-ee89-4167-9628-05705846e4ca\") " pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf"
Sep 17 21:11:45.831099 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.830970    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhczm\" (UniqueName: \"kubernetes.io/projected/e72d4ce2-ee89-4167-9628-05705846e4ca-kube-api-access-vhczm\") pod \"isvc-canary-create-v2-predictor-bfcdb48dd-jggsf\" (UID: \"e72d4ce2-ee89-4167-9628-05705846e4ca\") " pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf"
Sep 17 21:11:45.831099 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.831006    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e72d4ce2-ee89-4167-9628-05705846e4ca-kserve-provision-location\") pod \"isvc-canary-create-v2-predictor-bfcdb48dd-jggsf\" (UID: \"e72d4ce2-ee89-4167-9628-05705846e4ca\") " pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf"
Sep 17 21:11:45.831099 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.831023    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e72d4ce2-ee89-4167-9628-05705846e4ca-cabundle-cert\") pod \"isvc-canary-create-v2-predictor-bfcdb48dd-jggsf\" (UID: \"e72d4ce2-ee89-4167-9628-05705846e4ca\") " pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf"
Sep 17 21:11:45.843732 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.843702    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp"]
Sep 17 21:11:45.847222 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.847204    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp"
Sep 17 21:11:45.852417 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.852396    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-create-predictor-serving-cert\""
Sep 17 21:11:45.880721 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.880691    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp"]
Sep 17 21:11:45.920965 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.920867    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:11:45.931789 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.931746    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhczm\" (UniqueName: \"kubernetes.io/projected/e72d4ce2-ee89-4167-9628-05705846e4ca-kube-api-access-vhczm\") pod \"isvc-canary-create-v2-predictor-bfcdb48dd-jggsf\" (UID: \"e72d4ce2-ee89-4167-9628-05705846e4ca\") " pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf"
Sep 17 21:11:45.931942 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.931805    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e-cabundle-cert\") pod \"isvc-canary-create-predictor-7c7f45c769-cd7lp\" (UID: \"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp"
Sep 17 21:11:45.931942 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.931826    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e-proxy-tls\") pod \"isvc-canary-create-predictor-7c7f45c769-cd7lp\" (UID: \"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp"
Sep 17 21:11:45.931942 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.931858    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e72d4ce2-ee89-4167-9628-05705846e4ca-kserve-provision-location\") pod \"isvc-canary-create-v2-predictor-bfcdb48dd-jggsf\" (UID: \"e72d4ce2-ee89-4167-9628-05705846e4ca\") " pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf"
Sep 17 21:11:45.931942 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.931889    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e72d4ce2-ee89-4167-9628-05705846e4ca-cabundle-cert\") pod \"isvc-canary-create-v2-predictor-bfcdb48dd-jggsf\" (UID: \"e72d4ce2-ee89-4167-9628-05705846e4ca\") " pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf"
Sep 17 21:11:45.931942 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.931923    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e72d4ce2-ee89-4167-9628-05705846e4ca-proxy-tls\") pod \"isvc-canary-create-v2-predictor-bfcdb48dd-jggsf\" (UID: \"e72d4ce2-ee89-4167-9628-05705846e4ca\") " pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf"
Sep 17 21:11:45.932213 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.931947    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e72d4ce2-ee89-4167-9628-05705846e4ca-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-v2-predictor-bfcdb48dd-jggsf\" (UID: \"e72d4ce2-ee89-4167-9628-05705846e4ca\") " pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf"
Sep 17 21:11:45.932213 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.931966    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp848\" (UniqueName: \"kubernetes.io/projected/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e-kube-api-access-gp848\") pod \"isvc-canary-create-predictor-7c7f45c769-cd7lp\" (UID: \"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp"
Sep 17 21:11:45.932213 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.931998    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e-kserve-provision-location\") pod \"isvc-canary-create-predictor-7c7f45c769-cd7lp\" (UID: \"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp"
Sep 17 21:11:45.932213 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.932023    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-7c7f45c769-cd7lp\" (UID: \"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp"
Sep 17 21:11:45.932383 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.932248    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e72d4ce2-ee89-4167-9628-05705846e4ca-kserve-provision-location\") pod \"isvc-canary-create-v2-predictor-bfcdb48dd-jggsf\" (UID: \"e72d4ce2-ee89-4167-9628-05705846e4ca\") " pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf"
Sep 17 21:11:45.932536 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.932514    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e72d4ce2-ee89-4167-9628-05705846e4ca-cabundle-cert\") pod \"isvc-canary-create-v2-predictor-bfcdb48dd-jggsf\" (UID: \"e72d4ce2-ee89-4167-9628-05705846e4ca\") " pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf"
Sep 17 21:11:45.932576 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.932526    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e72d4ce2-ee89-4167-9628-05705846e4ca-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-v2-predictor-bfcdb48dd-jggsf\" (UID: \"e72d4ce2-ee89-4167-9628-05705846e4ca\") " pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf"
Sep 17 21:11:45.934333 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.934316    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e72d4ce2-ee89-4167-9628-05705846e4ca-proxy-tls\") pod \"isvc-canary-create-v2-predictor-bfcdb48dd-jggsf\" (UID: \"e72d4ce2-ee89-4167-9628-05705846e4ca\") " pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf"
Sep 17 21:11:45.944515 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:45.944491    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhczm\" (UniqueName: \"kubernetes.io/projected/e72d4ce2-ee89-4167-9628-05705846e4ca-kube-api-access-vhczm\") pod \"isvc-canary-create-v2-predictor-bfcdb48dd-jggsf\" (UID: \"e72d4ce2-ee89-4167-9628-05705846e4ca\") " pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf"
Sep 17 21:11:46.030599 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:46.030566    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf"
Sep 17 21:11:46.032916 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:46.032894    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp848\" (UniqueName: \"kubernetes.io/projected/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e-kube-api-access-gp848\") pod \"isvc-canary-create-predictor-7c7f45c769-cd7lp\" (UID: \"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp"
Sep 17 21:11:46.033002 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:46.032927    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e-kserve-provision-location\") pod \"isvc-canary-create-predictor-7c7f45c769-cd7lp\" (UID: \"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp"
Sep 17 21:11:46.033002 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:46.032945    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-7c7f45c769-cd7lp\" (UID: \"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp"
Sep 17 21:11:46.033106 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:46.032994    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e-cabundle-cert\") pod \"isvc-canary-create-predictor-7c7f45c769-cd7lp\" (UID: \"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp"
Sep 17 21:11:46.033106 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:46.033032    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e-proxy-tls\") pod \"isvc-canary-create-predictor-7c7f45c769-cd7lp\" (UID: \"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp"
Sep 17 21:11:46.033340 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:46.033319    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e-kserve-provision-location\") pod \"isvc-canary-create-predictor-7c7f45c769-cd7lp\" (UID: \"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp"
Sep 17 21:11:46.033629 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:46.033608    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e-cabundle-cert\") pod \"isvc-canary-create-predictor-7c7f45c769-cd7lp\" (UID: \"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp"
Sep 17 21:11:46.033770 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:46.033625    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-7c7f45c769-cd7lp\" (UID: \"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp"
Sep 17 21:11:46.035461 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:46.035442    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e-proxy-tls\") pod \"isvc-canary-create-predictor-7c7f45c769-cd7lp\" (UID: \"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp"
Sep 17 21:11:46.057967 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:46.057944    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp848\" (UniqueName: \"kubernetes.io/projected/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e-kube-api-access-gp848\") pod \"isvc-canary-create-predictor-7c7f45c769-cd7lp\" (UID: \"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp"
Sep 17 21:11:46.074908 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:46.074874    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:11:46.144396 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:46.144363    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:11:46.157254 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:46.157228    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp"
Sep 17 21:11:46.166359 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:46.166268    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 17 21:11:46.308379 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:46.308353    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf"]
Sep 17 21:11:46.310586 ip-10-0-132-201 kubenswrapper[2567]: W0917 21:11:46.310554    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode72d4ce2_ee89_4167_9628_05705846e4ca.slice/crio-c885c0f0fe5600feb0c4caf70c9331bfeb07aaf07afce1beee9eb3eea00887ad WatchSource:0}: Error finding container c885c0f0fe5600feb0c4caf70c9331bfeb07aaf07afce1beee9eb3eea00887ad: Status 404 returned error can't find the container with id c885c0f0fe5600feb0c4caf70c9331bfeb07aaf07afce1beee9eb3eea00887ad
Sep 17 21:11:46.429393 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:46.429367    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp"]
Sep 17 21:11:46.431743 ip-10-0-132-201 kubenswrapper[2567]: W0917 21:11:46.431717    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f2ddaad_0579_48f5_a8d8_ced1f2a8976e.slice/crio-a6be69e05f7bcd9e5411e5736f7df7a7a060a67840b5686efa3da8d7728d8006 WatchSource:0}: Error finding container a6be69e05f7bcd9e5411e5736f7df7a7a060a67840b5686efa3da8d7728d8006: Status 404 returned error can't find the container with id a6be69e05f7bcd9e5411e5736f7df7a7a060a67840b5686efa3da8d7728d8006
Sep 17 21:11:46.674495 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:46.674459    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf" event={"ID":"e72d4ce2-ee89-4167-9628-05705846e4ca","Type":"ContainerStarted","Data":"ac8afd113f7c9dc3275565232e727dfe2dd796745080f55cff1b71cf430e30d8"}
Sep 17 21:11:46.674977 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:46.674561    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf" event={"ID":"e72d4ce2-ee89-4167-9628-05705846e4ca","Type":"ContainerStarted","Data":"c885c0f0fe5600feb0c4caf70c9331bfeb07aaf07afce1beee9eb3eea00887ad"}
Sep 17 21:11:46.676252 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:46.676222    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp" event={"ID":"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e","Type":"ContainerStarted","Data":"86db1a47d51db382e2f4f13588f0d251b610c59e5c7d767bb25ac1d0a7f6ad45"}
Sep 17 21:11:46.676360 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:46.676287    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp" event={"ID":"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e","Type":"ContainerStarted","Data":"a6be69e05f7bcd9e5411e5736f7df7a7a060a67840b5686efa3da8d7728d8006"}
Sep 17 21:11:46.997540 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:46.997502    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.31:8643/healthz\": dial tcp 10.132.0.31:8643: connect: connection refused"
Sep 17 21:11:50.694602 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:50.694517    2567 generic.go:356] "Generic (PLEG): container finished" podID="2f2ddaad-0579-48f5-a8d8-ced1f2a8976e" containerID="86db1a47d51db382e2f4f13588f0d251b610c59e5c7d767bb25ac1d0a7f6ad45" exitCode=0
Sep 17 21:11:50.694602 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:50.694588    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp" event={"ID":"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e","Type":"ContainerDied","Data":"86db1a47d51db382e2f4f13588f0d251b610c59e5c7d767bb25ac1d0a7f6ad45"}
Sep 17 21:11:50.696209 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:50.696188    2567 generic.go:356] "Generic (PLEG): container finished" podID="e72d4ce2-ee89-4167-9628-05705846e4ca" containerID="ac8afd113f7c9dc3275565232e727dfe2dd796745080f55cff1b71cf430e30d8" exitCode=0
Sep 17 21:11:50.696302 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:50.696231    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf" event={"ID":"e72d4ce2-ee89-4167-9628-05705846e4ca","Type":"ContainerDied","Data":"ac8afd113f7c9dc3275565232e727dfe2dd796745080f55cff1b71cf430e30d8"}
Sep 17 21:11:51.702396 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:51.702356    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf" event={"ID":"e72d4ce2-ee89-4167-9628-05705846e4ca","Type":"ContainerStarted","Data":"9218316b784653d82119d5a5e44b3e8cf0a2ca5d4810e80a94982a82d8d4fa23"}
Sep 17 21:11:51.702848 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:51.702457    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf" event={"ID":"e72d4ce2-ee89-4167-9628-05705846e4ca","Type":"ContainerStarted","Data":"cb5ac1ed8696c6271131c404182fea203ef74470603706f740d0b6a0e6b2bf41"}
Sep 17 21:11:51.702848 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:51.702633    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf"
Sep 17 21:11:51.704498 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:51.704473    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp" event={"ID":"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e","Type":"ContainerStarted","Data":"a200c8758d909bf84cc31accacb6be9050ee42d10d208ce8a3e85b639aa56174"}
Sep 17 21:11:51.704613 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:51.704528    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp" event={"ID":"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e","Type":"ContainerStarted","Data":"0663fb8a1b5052b32c5ce19654d4a6d11f9a7c2082cb3328e6b203465cac3907"}
Sep 17 21:11:51.704772 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:51.704729    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp"
Sep 17 21:11:51.722044 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:51.721996    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf" podStartSLOduration=2.9504936539999997 podStartE2EDuration="6.721981466s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.771487812s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:11:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:11:51.721700264 +0000 UTC m=+1376.067302142" watchObservedRunningTime="2026-09-17 21:11:51.721981466 +0000 UTC m=+1376.067583293"
Sep 17 21:11:51.741966 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:51.741914    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp" podStartSLOduration=2.823010391 podStartE2EDuration="6.741898884s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.918888493s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:11:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:11:51.740682162 +0000 UTC m=+1376.086284010" watchObservedRunningTime="2026-09-17 21:11:51.741898884 +0000 UTC m=+1376.087500752"
Sep 17 21:11:51.997269 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:51.997224    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.31:8643/healthz\": dial tcp 10.132.0.31:8643: connect: connection refused"
Sep 17 21:11:51.997440 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:51.997376    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"
Sep 17 21:11:52.709182 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:52.709149    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf"
Sep 17 21:11:52.709551 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:52.709204    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp"
Sep 17 21:11:52.710223 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:52.710192    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp" podUID="2f2ddaad-0579-48f5-a8d8-ced1f2a8976e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Sep 17 21:11:52.710275 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:52.710250    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf" podUID="e72d4ce2-ee89-4167-9628-05705846e4ca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Sep 17 21:11:53.713357 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:53.713309    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp" podUID="2f2ddaad-0579-48f5-a8d8-ced1f2a8976e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Sep 17 21:11:53.713864 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:53.713445    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf" podUID="e72d4ce2-ee89-4167-9628-05705846e4ca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Sep 17 21:11:54.190148 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.190123    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"
Sep 17 21:11:54.305546 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.305463    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3ad76ac7-1b45-4a8b-9472-a3902021e09f-kserve-provision-location\") pod \"3ad76ac7-1b45-4a8b-9472-a3902021e09f\" (UID: \"3ad76ac7-1b45-4a8b-9472-a3902021e09f\") "
Sep 17 21:11:54.305546 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.305505    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6sdth\" (UniqueName: \"kubernetes.io/projected/3ad76ac7-1b45-4a8b-9472-a3902021e09f-kube-api-access-6sdth\") pod \"3ad76ac7-1b45-4a8b-9472-a3902021e09f\" (UID: \"3ad76ac7-1b45-4a8b-9472-a3902021e09f\") "
Sep 17 21:11:54.305810 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.305577    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3ad76ac7-1b45-4a8b-9472-a3902021e09f-proxy-tls\") pod \"3ad76ac7-1b45-4a8b-9472-a3902021e09f\" (UID: \"3ad76ac7-1b45-4a8b-9472-a3902021e09f\") "
Sep 17 21:11:54.305810 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.305615    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3ad76ac7-1b45-4a8b-9472-a3902021e09f-kube-rbac-proxy-sar-config\") pod \"3ad76ac7-1b45-4a8b-9472-a3902021e09f\" (UID: \"3ad76ac7-1b45-4a8b-9472-a3902021e09f\") "
Sep 17 21:11:54.305810 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.305659    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3ad76ac7-1b45-4a8b-9472-a3902021e09f-cabundle-cert\") pod \"3ad76ac7-1b45-4a8b-9472-a3902021e09f\" (UID: \"3ad76ac7-1b45-4a8b-9472-a3902021e09f\") "
Sep 17 21:11:54.305810 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.305781    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ad76ac7-1b45-4a8b-9472-a3902021e09f-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "3ad76ac7-1b45-4a8b-9472-a3902021e09f" (UID: "3ad76ac7-1b45-4a8b-9472-a3902021e09f"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 21:11:54.306035 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.306015    2567 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3ad76ac7-1b45-4a8b-9472-a3902021e09f-kserve-provision-location\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:11:54.306074 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.306034    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ad76ac7-1b45-4a8b-9472-a3902021e09f-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "3ad76ac7-1b45-4a8b-9472-a3902021e09f" (UID: "3ad76ac7-1b45-4a8b-9472-a3902021e09f"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:11:54.306074 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.306052    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ad76ac7-1b45-4a8b-9472-a3902021e09f-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "3ad76ac7-1b45-4a8b-9472-a3902021e09f" (UID: "3ad76ac7-1b45-4a8b-9472-a3902021e09f"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:11:54.307621 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.307596    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ad76ac7-1b45-4a8b-9472-a3902021e09f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "3ad76ac7-1b45-4a8b-9472-a3902021e09f" (UID: "3ad76ac7-1b45-4a8b-9472-a3902021e09f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:11:54.307727 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.307652    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ad76ac7-1b45-4a8b-9472-a3902021e09f-kube-api-access-6sdth" (OuterVolumeSpecName: "kube-api-access-6sdth") pod "3ad76ac7-1b45-4a8b-9472-a3902021e09f" (UID: "3ad76ac7-1b45-4a8b-9472-a3902021e09f"). InnerVolumeSpecName "kube-api-access-6sdth". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:11:54.406444 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.406411    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3ad76ac7-1b45-4a8b-9472-a3902021e09f-proxy-tls\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:11:54.406444 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.406439    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3ad76ac7-1b45-4a8b-9472-a3902021e09f-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:11:54.406444 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.406448    2567 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3ad76ac7-1b45-4a8b-9472-a3902021e09f-cabundle-cert\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:11:54.406665 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.406456    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6sdth\" (UniqueName: \"kubernetes.io/projected/3ad76ac7-1b45-4a8b-9472-a3902021e09f-kube-api-access-6sdth\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:11:54.719935 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.719906    2567 generic.go:356] "Generic (PLEG): container finished" podID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerID="1a248d32bb0b0575b14af02bf6ffd033872a834ed8a2306d76068f38d4f0a7bf" exitCode=0
Sep 17 21:11:54.720320 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.719954    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" event={"ID":"3ad76ac7-1b45-4a8b-9472-a3902021e09f","Type":"ContainerDied","Data":"1a248d32bb0b0575b14af02bf6ffd033872a834ed8a2306d76068f38d4f0a7bf"}
Sep 17 21:11:54.720320 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.719968    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"
Sep 17 21:11:54.720320 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.719999    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7" event={"ID":"3ad76ac7-1b45-4a8b-9472-a3902021e09f","Type":"ContainerDied","Data":"985acdfb6dbb9bbe24b2ecdf96e3016c9fb7fe3e9c843f8c028994754397cb0a"}
Sep 17 21:11:54.720320 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.720027    2567 scope.go:117] "RemoveContainer" containerID="077b77af4efa0f71394430fa9b88839bb9071837eff118b9bb5045bb00f80d2f"
Sep 17 21:11:54.728596 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.728577    2567 scope.go:117] "RemoveContainer" containerID="1a248d32bb0b0575b14af02bf6ffd033872a834ed8a2306d76068f38d4f0a7bf"
Sep 17 21:11:54.736432 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.736410    2567 scope.go:117] "RemoveContainer" containerID="88ec544f3ca4ead1615a823f9a0ead28116b16b8c5c9ee412139f9f272047b10"
Sep 17 21:11:54.744977 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.744960    2567 scope.go:117] "RemoveContainer" containerID="077b77af4efa0f71394430fa9b88839bb9071837eff118b9bb5045bb00f80d2f"
Sep 17 21:11:54.745298 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:11:54.745270    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"077b77af4efa0f71394430fa9b88839bb9071837eff118b9bb5045bb00f80d2f\": container with ID starting with 077b77af4efa0f71394430fa9b88839bb9071837eff118b9bb5045bb00f80d2f not found: ID does not exist" containerID="077b77af4efa0f71394430fa9b88839bb9071837eff118b9bb5045bb00f80d2f"
Sep 17 21:11:54.745403 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.745306    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"077b77af4efa0f71394430fa9b88839bb9071837eff118b9bb5045bb00f80d2f"} err="failed to get container status \"077b77af4efa0f71394430fa9b88839bb9071837eff118b9bb5045bb00f80d2f\": rpc error: code = NotFound desc = could not find container \"077b77af4efa0f71394430fa9b88839bb9071837eff118b9bb5045bb00f80d2f\": container with ID starting with 077b77af4efa0f71394430fa9b88839bb9071837eff118b9bb5045bb00f80d2f not found: ID does not exist"
Sep 17 21:11:54.745403 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.745326    2567 scope.go:117] "RemoveContainer" containerID="1a248d32bb0b0575b14af02bf6ffd033872a834ed8a2306d76068f38d4f0a7bf"
Sep 17 21:11:54.745520 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.745419    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"]
Sep 17 21:11:54.745621 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:11:54.745603    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a248d32bb0b0575b14af02bf6ffd033872a834ed8a2306d76068f38d4f0a7bf\": container with ID starting with 1a248d32bb0b0575b14af02bf6ffd033872a834ed8a2306d76068f38d4f0a7bf not found: ID does not exist" containerID="1a248d32bb0b0575b14af02bf6ffd033872a834ed8a2306d76068f38d4f0a7bf"
Sep 17 21:11:54.745664 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.745627    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a248d32bb0b0575b14af02bf6ffd033872a834ed8a2306d76068f38d4f0a7bf"} err="failed to get container status \"1a248d32bb0b0575b14af02bf6ffd033872a834ed8a2306d76068f38d4f0a7bf\": rpc error: code = NotFound desc = could not find container \"1a248d32bb0b0575b14af02bf6ffd033872a834ed8a2306d76068f38d4f0a7bf\": container with ID starting with 1a248d32bb0b0575b14af02bf6ffd033872a834ed8a2306d76068f38d4f0a7bf not found: ID does not exist"
Sep 17 21:11:54.745664 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.745642    2567 scope.go:117] "RemoveContainer" containerID="88ec544f3ca4ead1615a823f9a0ead28116b16b8c5c9ee412139f9f272047b10"
Sep 17 21:11:54.745936 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:11:54.745918    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88ec544f3ca4ead1615a823f9a0ead28116b16b8c5c9ee412139f9f272047b10\": container with ID starting with 88ec544f3ca4ead1615a823f9a0ead28116b16b8c5c9ee412139f9f272047b10 not found: ID does not exist" containerID="88ec544f3ca4ead1615a823f9a0ead28116b16b8c5c9ee412139f9f272047b10"
Sep 17 21:11:54.746008 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.745944    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88ec544f3ca4ead1615a823f9a0ead28116b16b8c5c9ee412139f9f272047b10"} err="failed to get container status \"88ec544f3ca4ead1615a823f9a0ead28116b16b8c5c9ee412139f9f272047b10\": rpc error: code = NotFound desc = could not find container \"88ec544f3ca4ead1615a823f9a0ead28116b16b8c5c9ee412139f9f272047b10\": container with ID starting with 88ec544f3ca4ead1615a823f9a0ead28116b16b8c5c9ee412139f9f272047b10 not found: ID does not exist"
Sep 17 21:11:54.748803 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:54.748782    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-scale-raw-f9ecc-predictor-5df6567f54-zx8s7"]
Sep 17 21:11:56.168901 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:56.168867    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" path="/var/lib/kubelet/pods/3ad76ac7-1b45-4a8b-9472-a3902021e09f/volumes"
Sep 17 21:11:58.717826 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:58.717791    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf"
Sep 17 21:11:58.718251 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:58.718179    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp"
Sep 17 21:11:58.718368 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:58.718341    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf" podUID="e72d4ce2-ee89-4167-9628-05705846e4ca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Sep 17 21:11:58.718709 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:11:58.718681    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp" podUID="2f2ddaad-0579-48f5-a8d8-ced1f2a8976e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Sep 17 21:12:03.690786 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:12:03.690731    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:12:08.718979 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:12:08.718942    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf" podUID="e72d4ce2-ee89-4167-9628-05705846e4ca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Sep 17 21:12:08.719353 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:12:08.718942    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp" podUID="2f2ddaad-0579-48f5-a8d8-ced1f2a8976e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Sep 17 21:12:18.719187 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:12:18.719145    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf" podUID="e72d4ce2-ee89-4167-9628-05705846e4ca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Sep 17 21:12:18.719669 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:12:18.719148    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp" podUID="2f2ddaad-0579-48f5-a8d8-ced1f2a8976e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Sep 17 21:12:28.718980 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:12:28.718936    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf" podUID="e72d4ce2-ee89-4167-9628-05705846e4ca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Sep 17 21:12:28.719366 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:12:28.718944    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp" podUID="2f2ddaad-0579-48f5-a8d8-ced1f2a8976e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Sep 17 21:12:38.719109 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:12:38.719068    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp" podUID="2f2ddaad-0579-48f5-a8d8-ced1f2a8976e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Sep 17 21:12:38.719473 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:12:38.719069    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf" podUID="e72d4ce2-ee89-4167-9628-05705846e4ca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Sep 17 21:12:48.718353 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:12:48.718253    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf" podUID="e72d4ce2-ee89-4167-9628-05705846e4ca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Sep 17 21:12:48.718819 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:12:48.718554    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp" podUID="2f2ddaad-0579-48f5-a8d8-ced1f2a8976e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Sep 17 21:12:58.719279 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:12:58.719234    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp" podUID="2f2ddaad-0579-48f5-a8d8-ced1f2a8976e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Sep 17 21:12:58.719781 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:12:58.719236    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf" podUID="e72d4ce2-ee89-4167-9628-05705846e4ca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Sep 17 21:13:08.718771 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:08.718718    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf" podUID="e72d4ce2-ee89-4167-9628-05705846e4ca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Sep 17 21:13:08.719302 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:08.718899    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp"
Sep 17 21:13:18.719416 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:18.719384    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf"
Sep 17 21:13:18.849908 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:18.849873    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp"]
Sep 17 21:13:18.850153 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:18.850110    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp" podUID="2f2ddaad-0579-48f5-a8d8-ced1f2a8976e" containerName="kserve-container" containerID="cri-o://0663fb8a1b5052b32c5ce19654d4a6d11f9a7c2082cb3328e6b203465cac3907" gracePeriod=30
Sep 17 21:13:18.850216 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:18.850189    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp" podUID="2f2ddaad-0579-48f5-a8d8-ced1f2a8976e" containerName="kube-rbac-proxy" containerID="cri-o://a200c8758d909bf84cc31accacb6be9050ee42d10d208ce8a3e85b639aa56174" gracePeriod=30
Sep 17 21:13:19.049802 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:19.049703    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:13:19.076788 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:19.076744    2567 generic.go:356] "Generic (PLEG): container finished" podID="2f2ddaad-0579-48f5-a8d8-ced1f2a8976e" containerID="a200c8758d909bf84cc31accacb6be9050ee42d10d208ce8a3e85b639aa56174" exitCode=2
Sep 17 21:13:19.076929 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:19.076822    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp" event={"ID":"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e","Type":"ContainerDied","Data":"a200c8758d909bf84cc31accacb6be9050ee42d10d208ce8a3e85b639aa56174"}
Sep 17 21:13:20.865821 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:20.865788    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf"]
Sep 17 21:13:20.866207 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:20.866003    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf" podUID="e72d4ce2-ee89-4167-9628-05705846e4ca" containerName="kserve-container" containerID="cri-o://cb5ac1ed8696c6271131c404182fea203ef74470603706f740d0b6a0e6b2bf41" gracePeriod=30
Sep 17 21:13:20.866207 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:20.866035    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf" podUID="e72d4ce2-ee89-4167-9628-05705846e4ca" containerName="kube-rbac-proxy" containerID="cri-o://9218316b784653d82119d5a5e44b3e8cf0a2ca5d4810e80a94982a82d8d4fa23" gracePeriod=30
Sep 17 21:13:20.901069 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:20.901032    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:13:21.086932 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:21.086901    2567 generic.go:356] "Generic (PLEG): container finished" podID="e72d4ce2-ee89-4167-9628-05705846e4ca" containerID="9218316b784653d82119d5a5e44b3e8cf0a2ca5d4810e80a94982a82d8d4fa23" exitCode=2
Sep 17 21:13:21.087098 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:21.086990    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf" event={"ID":"e72d4ce2-ee89-4167-9628-05705846e4ca","Type":"ContainerDied","Data":"9218316b784653d82119d5a5e44b3e8cf0a2ca5d4810e80a94982a82d8d4fa23"}
Sep 17 21:13:23.713714 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:23.713670    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf" podUID="e72d4ce2-ee89-4167-9628-05705846e4ca" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.32:8643/healthz\": dial tcp 10.132.0.32:8643: connect: connection refused"
Sep 17 21:13:23.714179 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:23.713675    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp" podUID="2f2ddaad-0579-48f5-a8d8-ced1f2a8976e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.33:8643/healthz\": dial tcp 10.132.0.33:8643: connect: connection refused"
Sep 17 21:13:23.789346 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:23.789320    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp"
Sep 17 21:13:23.848730 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:23.848702    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gp848\" (UniqueName: \"kubernetes.io/projected/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e-kube-api-access-gp848\") pod \"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e\" (UID: \"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e\") "
Sep 17 21:13:23.848883 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:23.848734    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e-cabundle-cert\") pod \"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e\" (UID: \"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e\") "
Sep 17 21:13:23.848883 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:23.848789    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e-kserve-provision-location\") pod \"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e\" (UID: \"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e\") "
Sep 17 21:13:23.848883 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:23.848815    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e-kube-rbac-proxy-sar-config\") pod \"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e\" (UID: \"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e\") "
Sep 17 21:13:23.848883 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:23.848854    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e-proxy-tls\") pod \"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e\" (UID: \"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e\") "
Sep 17 21:13:23.849184 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:23.849137    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "2f2ddaad-0579-48f5-a8d8-ced1f2a8976e" (UID: "2f2ddaad-0579-48f5-a8d8-ced1f2a8976e"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 21:13:23.849184 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:23.849166    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "2f2ddaad-0579-48f5-a8d8-ced1f2a8976e" (UID: "2f2ddaad-0579-48f5-a8d8-ced1f2a8976e"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:13:23.849271 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:23.849196    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "2f2ddaad-0579-48f5-a8d8-ced1f2a8976e" (UID: "2f2ddaad-0579-48f5-a8d8-ced1f2a8976e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:13:23.850918 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:23.850885    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "2f2ddaad-0579-48f5-a8d8-ced1f2a8976e" (UID: "2f2ddaad-0579-48f5-a8d8-ced1f2a8976e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:13:23.851023 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:23.850945    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e-kube-api-access-gp848" (OuterVolumeSpecName: "kube-api-access-gp848") pod "2f2ddaad-0579-48f5-a8d8-ced1f2a8976e" (UID: "2f2ddaad-0579-48f5-a8d8-ced1f2a8976e"). InnerVolumeSpecName "kube-api-access-gp848". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:13:23.949398 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:23.949315    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gp848\" (UniqueName: \"kubernetes.io/projected/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e-kube-api-access-gp848\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:13:23.949398 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:23.949346    2567 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e-cabundle-cert\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:13:23.949398 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:23.949356    2567 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e-kserve-provision-location\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:13:23.949398 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:23.949377    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:13:23.949398 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:23.949388    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e-proxy-tls\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:13:24.101685 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:24.101648    2567 generic.go:356] "Generic (PLEG): container finished" podID="2f2ddaad-0579-48f5-a8d8-ced1f2a8976e" containerID="0663fb8a1b5052b32c5ce19654d4a6d11f9a7c2082cb3328e6b203465cac3907" exitCode=0
Sep 17 21:13:24.101948 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:24.101708    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp"
Sep 17 21:13:24.101948 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:24.101730    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp" event={"ID":"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e","Type":"ContainerDied","Data":"0663fb8a1b5052b32c5ce19654d4a6d11f9a7c2082cb3328e6b203465cac3907"}
Sep 17 21:13:24.101948 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:24.101819    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp" event={"ID":"2f2ddaad-0579-48f5-a8d8-ced1f2a8976e","Type":"ContainerDied","Data":"a6be69e05f7bcd9e5411e5736f7df7a7a060a67840b5686efa3da8d7728d8006"}
Sep 17 21:13:24.101948 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:24.101859    2567 scope.go:117] "RemoveContainer" containerID="a200c8758d909bf84cc31accacb6be9050ee42d10d208ce8a3e85b639aa56174"
Sep 17 21:13:24.110273 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:24.110255    2567 scope.go:117] "RemoveContainer" containerID="0663fb8a1b5052b32c5ce19654d4a6d11f9a7c2082cb3328e6b203465cac3907"
Sep 17 21:13:24.117500 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:24.117481    2567 scope.go:117] "RemoveContainer" containerID="86db1a47d51db382e2f4f13588f0d251b610c59e5c7d767bb25ac1d0a7f6ad45"
Sep 17 21:13:24.124146 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:24.124120    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp"]
Sep 17 21:13:24.124673 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:24.124659    2567 scope.go:117] "RemoveContainer" containerID="a200c8758d909bf84cc31accacb6be9050ee42d10d208ce8a3e85b639aa56174"
Sep 17 21:13:24.124990 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:13:24.124971    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a200c8758d909bf84cc31accacb6be9050ee42d10d208ce8a3e85b639aa56174\": container with ID starting with a200c8758d909bf84cc31accacb6be9050ee42d10d208ce8a3e85b639aa56174 not found: ID does not exist" containerID="a200c8758d909bf84cc31accacb6be9050ee42d10d208ce8a3e85b639aa56174"
Sep 17 21:13:24.125059 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:24.125000    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a200c8758d909bf84cc31accacb6be9050ee42d10d208ce8a3e85b639aa56174"} err="failed to get container status \"a200c8758d909bf84cc31accacb6be9050ee42d10d208ce8a3e85b639aa56174\": rpc error: code = NotFound desc = could not find container \"a200c8758d909bf84cc31accacb6be9050ee42d10d208ce8a3e85b639aa56174\": container with ID starting with a200c8758d909bf84cc31accacb6be9050ee42d10d208ce8a3e85b639aa56174 not found: ID does not exist"
Sep 17 21:13:24.125059 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:24.125019    2567 scope.go:117] "RemoveContainer" containerID="0663fb8a1b5052b32c5ce19654d4a6d11f9a7c2082cb3328e6b203465cac3907"
Sep 17 21:13:24.125282 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:13:24.125265    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0663fb8a1b5052b32c5ce19654d4a6d11f9a7c2082cb3328e6b203465cac3907\": container with ID starting with 0663fb8a1b5052b32c5ce19654d4a6d11f9a7c2082cb3328e6b203465cac3907 not found: ID does not exist" containerID="0663fb8a1b5052b32c5ce19654d4a6d11f9a7c2082cb3328e6b203465cac3907"
Sep 17 21:13:24.125326 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:24.125288    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0663fb8a1b5052b32c5ce19654d4a6d11f9a7c2082cb3328e6b203465cac3907"} err="failed to get container status \"0663fb8a1b5052b32c5ce19654d4a6d11f9a7c2082cb3328e6b203465cac3907\": rpc error: code = NotFound desc = could not find container \"0663fb8a1b5052b32c5ce19654d4a6d11f9a7c2082cb3328e6b203465cac3907\": container with ID starting with 0663fb8a1b5052b32c5ce19654d4a6d11f9a7c2082cb3328e6b203465cac3907 not found: ID does not exist"
Sep 17 21:13:24.125326 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:24.125304    2567 scope.go:117] "RemoveContainer" containerID="86db1a47d51db382e2f4f13588f0d251b610c59e5c7d767bb25ac1d0a7f6ad45"
Sep 17 21:13:24.125543 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:13:24.125528    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86db1a47d51db382e2f4f13588f0d251b610c59e5c7d767bb25ac1d0a7f6ad45\": container with ID starting with 86db1a47d51db382e2f4f13588f0d251b610c59e5c7d767bb25ac1d0a7f6ad45 not found: ID does not exist" containerID="86db1a47d51db382e2f4f13588f0d251b610c59e5c7d767bb25ac1d0a7f6ad45"
Sep 17 21:13:24.125595 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:24.125546    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86db1a47d51db382e2f4f13588f0d251b610c59e5c7d767bb25ac1d0a7f6ad45"} err="failed to get container status \"86db1a47d51db382e2f4f13588f0d251b610c59e5c7d767bb25ac1d0a7f6ad45\": rpc error: code = NotFound desc = could not find container \"86db1a47d51db382e2f4f13588f0d251b610c59e5c7d767bb25ac1d0a7f6ad45\": container with ID starting with 86db1a47d51db382e2f4f13588f0d251b610c59e5c7d767bb25ac1d0a7f6ad45 not found: ID does not exist"
Sep 17 21:13:24.128222 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:24.128200    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-create-predictor-7c7f45c769-cd7lp"]
Sep 17 21:13:24.169018 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:24.168993    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f2ddaad-0579-48f5-a8d8-ced1f2a8976e" path="/var/lib/kubelet/pods/2f2ddaad-0579-48f5-a8d8-ced1f2a8976e/volumes"
Sep 17 21:13:25.714186 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.714162    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf"
Sep 17 21:13:25.759812 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.759788    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e72d4ce2-ee89-4167-9628-05705846e4ca-proxy-tls\") pod \"e72d4ce2-ee89-4167-9628-05705846e4ca\" (UID: \"e72d4ce2-ee89-4167-9628-05705846e4ca\") "
Sep 17 21:13:25.759953 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.759836    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e72d4ce2-ee89-4167-9628-05705846e4ca-kserve-provision-location\") pod \"e72d4ce2-ee89-4167-9628-05705846e4ca\" (UID: \"e72d4ce2-ee89-4167-9628-05705846e4ca\") "
Sep 17 21:13:25.759953 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.759875    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhczm\" (UniqueName: \"kubernetes.io/projected/e72d4ce2-ee89-4167-9628-05705846e4ca-kube-api-access-vhczm\") pod \"e72d4ce2-ee89-4167-9628-05705846e4ca\" (UID: \"e72d4ce2-ee89-4167-9628-05705846e4ca\") "
Sep 17 21:13:25.759953 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.759892    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e72d4ce2-ee89-4167-9628-05705846e4ca-cabundle-cert\") pod \"e72d4ce2-ee89-4167-9628-05705846e4ca\" (UID: \"e72d4ce2-ee89-4167-9628-05705846e4ca\") "
Sep 17 21:13:25.759953 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.759931    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e72d4ce2-ee89-4167-9628-05705846e4ca-kube-rbac-proxy-sar-config\") pod \"e72d4ce2-ee89-4167-9628-05705846e4ca\" (UID: \"e72d4ce2-ee89-4167-9628-05705846e4ca\") "
Sep 17 21:13:25.760201 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.760176    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e72d4ce2-ee89-4167-9628-05705846e4ca-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "e72d4ce2-ee89-4167-9628-05705846e4ca" (UID: "e72d4ce2-ee89-4167-9628-05705846e4ca"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 21:13:25.760267 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.760215    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e72d4ce2-ee89-4167-9628-05705846e4ca-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "e72d4ce2-ee89-4167-9628-05705846e4ca" (UID: "e72d4ce2-ee89-4167-9628-05705846e4ca"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:13:25.760267 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.760251    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e72d4ce2-ee89-4167-9628-05705846e4ca-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "e72d4ce2-ee89-4167-9628-05705846e4ca" (UID: "e72d4ce2-ee89-4167-9628-05705846e4ca"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:13:25.761960 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.761935    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e72d4ce2-ee89-4167-9628-05705846e4ca-kube-api-access-vhczm" (OuterVolumeSpecName: "kube-api-access-vhczm") pod "e72d4ce2-ee89-4167-9628-05705846e4ca" (UID: "e72d4ce2-ee89-4167-9628-05705846e4ca"). InnerVolumeSpecName "kube-api-access-vhczm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:13:25.762059 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.761966    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e72d4ce2-ee89-4167-9628-05705846e4ca-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "e72d4ce2-ee89-4167-9628-05705846e4ca" (UID: "e72d4ce2-ee89-4167-9628-05705846e4ca"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:13:25.861018 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.860932    2567 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e72d4ce2-ee89-4167-9628-05705846e4ca-kserve-provision-location\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:13:25.861018 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.860962    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhczm\" (UniqueName: \"kubernetes.io/projected/e72d4ce2-ee89-4167-9628-05705846e4ca-kube-api-access-vhczm\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:13:25.861018 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.860978    2567 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e72d4ce2-ee89-4167-9628-05705846e4ca-cabundle-cert\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:13:25.861018 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.860990    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e72d4ce2-ee89-4167-9628-05705846e4ca-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:13:25.861018 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.861005    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e72d4ce2-ee89-4167-9628-05705846e4ca-proxy-tls\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:13:25.873799 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.873742    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:13:25.945170 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.945138    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt"]
Sep 17 21:13:25.945472 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.945459    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e72d4ce2-ee89-4167-9628-05705846e4ca" containerName="kube-rbac-proxy"
Sep 17 21:13:25.945517 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.945474    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e72d4ce2-ee89-4167-9628-05705846e4ca" containerName="kube-rbac-proxy"
Sep 17 21:13:25.945517 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.945489    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e72d4ce2-ee89-4167-9628-05705846e4ca" containerName="storage-initializer"
Sep 17 21:13:25.945517 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.945497    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e72d4ce2-ee89-4167-9628-05705846e4ca" containerName="storage-initializer"
Sep 17 21:13:25.945517 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.945510    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="storage-initializer"
Sep 17 21:13:25.945517 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.945515    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="storage-initializer"
Sep 17 21:13:25.945688 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.945521    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f2ddaad-0579-48f5-a8d8-ced1f2a8976e" containerName="storage-initializer"
Sep 17 21:13:25.945688 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.945526    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f2ddaad-0579-48f5-a8d8-ced1f2a8976e" containerName="storage-initializer"
Sep 17 21:13:25.945688 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.945535    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kube-rbac-proxy"
Sep 17 21:13:25.945688 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.945540    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kube-rbac-proxy"
Sep 17 21:13:25.945688 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.945547    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e72d4ce2-ee89-4167-9628-05705846e4ca" containerName="kserve-container"
Sep 17 21:13:25.945688 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.945552    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e72d4ce2-ee89-4167-9628-05705846e4ca" containerName="kserve-container"
Sep 17 21:13:25.945688 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.945560    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f2ddaad-0579-48f5-a8d8-ced1f2a8976e" containerName="kube-rbac-proxy"
Sep 17 21:13:25.945688 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.945569    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f2ddaad-0579-48f5-a8d8-ced1f2a8976e" containerName="kube-rbac-proxy"
Sep 17 21:13:25.945688 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.945586    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kserve-container"
Sep 17 21:13:25.945688 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.945591    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kserve-container"
Sep 17 21:13:25.945688 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.945598    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f2ddaad-0579-48f5-a8d8-ced1f2a8976e" containerName="kserve-container"
Sep 17 21:13:25.945688 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.945604    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f2ddaad-0579-48f5-a8d8-ced1f2a8976e" containerName="kserve-container"
Sep 17 21:13:25.945688 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.945658    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e72d4ce2-ee89-4167-9628-05705846e4ca" containerName="kserve-container"
Sep 17 21:13:25.945688 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.945668    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f2ddaad-0579-48f5-a8d8-ced1f2a8976e" containerName="kserve-container"
Sep 17 21:13:25.945688 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.945675    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f2ddaad-0579-48f5-a8d8-ced1f2a8976e" containerName="kube-rbac-proxy"
Sep 17 21:13:25.945688 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.945682    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kube-rbac-proxy"
Sep 17 21:13:25.945688 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.945686    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ad76ac7-1b45-4a8b-9472-a3902021e09f" containerName="kserve-container"
Sep 17 21:13:25.945688 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.945692    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e72d4ce2-ee89-4167-9628-05705846e4ca" containerName="kube-rbac-proxy"
Sep 17 21:13:25.949222 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.949205    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt"
Sep 17 21:13:25.952001 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.951968    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-promote-predictor-serving-cert\""
Sep 17 21:13:25.952129 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.952003    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-promote-kube-rbac-proxy-sar-config\""
Sep 17 21:13:25.961375 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.961350    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ffb8954a-ca80-459c-974e-0d549edd1e6c-kserve-provision-location\") pod \"isvc-canary-promote-predictor-78b4c78fb9-8pzzt\" (UID: \"ffb8954a-ca80-459c-974e-0d549edd1e6c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt"
Sep 17 21:13:25.961490 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.961400    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ffb8954a-ca80-459c-974e-0d549edd1e6c-proxy-tls\") pod \"isvc-canary-promote-predictor-78b4c78fb9-8pzzt\" (UID: \"ffb8954a-ca80-459c-974e-0d549edd1e6c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt"
Sep 17 21:13:25.961490 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.961477    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ffb8954a-ca80-459c-974e-0d549edd1e6c-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-78b4c78fb9-8pzzt\" (UID: \"ffb8954a-ca80-459c-974e-0d549edd1e6c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt"
Sep 17 21:13:25.961633 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.961542    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p62bh\" (UniqueName: \"kubernetes.io/projected/ffb8954a-ca80-459c-974e-0d549edd1e6c-kube-api-access-p62bh\") pod \"isvc-canary-promote-predictor-78b4c78fb9-8pzzt\" (UID: \"ffb8954a-ca80-459c-974e-0d549edd1e6c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt"
Sep 17 21:13:25.961633 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.961604    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ffb8954a-ca80-459c-974e-0d549edd1e6c-cabundle-cert\") pod \"isvc-canary-promote-predictor-78b4c78fb9-8pzzt\" (UID: \"ffb8954a-ca80-459c-974e-0d549edd1e6c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt"
Sep 17 21:13:25.965485 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.965464    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt"]
Sep 17 21:13:25.978311 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:25.978289    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:13:26.061984 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:26.061952    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ffb8954a-ca80-459c-974e-0d549edd1e6c-proxy-tls\") pod \"isvc-canary-promote-predictor-78b4c78fb9-8pzzt\" (UID: \"ffb8954a-ca80-459c-974e-0d549edd1e6c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt"
Sep 17 21:13:26.062179 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:26.061995    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ffb8954a-ca80-459c-974e-0d549edd1e6c-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-78b4c78fb9-8pzzt\" (UID: \"ffb8954a-ca80-459c-974e-0d549edd1e6c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt"
Sep 17 21:13:26.062179 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:26.062037    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p62bh\" (UniqueName: \"kubernetes.io/projected/ffb8954a-ca80-459c-974e-0d549edd1e6c-kube-api-access-p62bh\") pod \"isvc-canary-promote-predictor-78b4c78fb9-8pzzt\" (UID: \"ffb8954a-ca80-459c-974e-0d549edd1e6c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt"
Sep 17 21:13:26.062179 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:26.062073    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ffb8954a-ca80-459c-974e-0d549edd1e6c-cabundle-cert\") pod \"isvc-canary-promote-predictor-78b4c78fb9-8pzzt\" (UID: \"ffb8954a-ca80-459c-974e-0d549edd1e6c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt"
Sep 17 21:13:26.062179 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:26.062099    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ffb8954a-ca80-459c-974e-0d549edd1e6c-kserve-provision-location\") pod \"isvc-canary-promote-predictor-78b4c78fb9-8pzzt\" (UID: \"ffb8954a-ca80-459c-974e-0d549edd1e6c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt"
Sep 17 21:13:26.062179 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:13:26.062101    2567 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-canary-promote-predictor-serving-cert: secret "isvc-canary-promote-predictor-serving-cert" not found
Sep 17 21:13:26.062179 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:13:26.062170    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffb8954a-ca80-459c-974e-0d549edd1e6c-proxy-tls podName:ffb8954a-ca80-459c-974e-0d549edd1e6c nodeName:}" failed. No retries permitted until 2026-09-17 21:13:26.562154197 +0000 UTC m=+1470.907756005 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/ffb8954a-ca80-459c-974e-0d549edd1e6c-proxy-tls") pod "isvc-canary-promote-predictor-78b4c78fb9-8pzzt" (UID: "ffb8954a-ca80-459c-974e-0d549edd1e6c") : secret "isvc-canary-promote-predictor-serving-cert" not found
Sep 17 21:13:26.064178 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:26.064149    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ffb8954a-ca80-459c-974e-0d549edd1e6c-kserve-provision-location\") pod \"isvc-canary-promote-predictor-78b4c78fb9-8pzzt\" (UID: \"ffb8954a-ca80-459c-974e-0d549edd1e6c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt"
Sep 17 21:13:26.065219 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:26.065197    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ffb8954a-ca80-459c-974e-0d549edd1e6c-cabundle-cert\") pod \"isvc-canary-promote-predictor-78b4c78fb9-8pzzt\" (UID: \"ffb8954a-ca80-459c-974e-0d549edd1e6c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt"
Sep 17 21:13:26.065296 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:26.065216    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ffb8954a-ca80-459c-974e-0d549edd1e6c-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-78b4c78fb9-8pzzt\" (UID: \"ffb8954a-ca80-459c-974e-0d549edd1e6c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt"
Sep 17 21:13:26.076797 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:26.076749    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p62bh\" (UniqueName: \"kubernetes.io/projected/ffb8954a-ca80-459c-974e-0d549edd1e6c-kube-api-access-p62bh\") pod \"isvc-canary-promote-predictor-78b4c78fb9-8pzzt\" (UID: \"ffb8954a-ca80-459c-974e-0d549edd1e6c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt"
Sep 17 21:13:26.081214 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:26.081186    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:13:26.118156 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:26.118080    2567 generic.go:356] "Generic (PLEG): container finished" podID="e72d4ce2-ee89-4167-9628-05705846e4ca" containerID="cb5ac1ed8696c6271131c404182fea203ef74470603706f740d0b6a0e6b2bf41" exitCode=0
Sep 17 21:13:26.118156 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:26.118147    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf" event={"ID":"e72d4ce2-ee89-4167-9628-05705846e4ca","Type":"ContainerDied","Data":"cb5ac1ed8696c6271131c404182fea203ef74470603706f740d0b6a0e6b2bf41"}
Sep 17 21:13:26.118325 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:26.118215    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf" event={"ID":"e72d4ce2-ee89-4167-9628-05705846e4ca","Type":"ContainerDied","Data":"c885c0f0fe5600feb0c4caf70c9331bfeb07aaf07afce1beee9eb3eea00887ad"}
Sep 17 21:13:26.118325 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:26.118147    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf"
Sep 17 21:13:26.118325 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:26.118239    2567 scope.go:117] "RemoveContainer" containerID="9218316b784653d82119d5a5e44b3e8cf0a2ca5d4810e80a94982a82d8d4fa23"
Sep 17 21:13:26.126904 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:26.126886    2567 scope.go:117] "RemoveContainer" containerID="cb5ac1ed8696c6271131c404182fea203ef74470603706f740d0b6a0e6b2bf41"
Sep 17 21:13:26.136189 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:26.136174    2567 scope.go:117] "RemoveContainer" containerID="ac8afd113f7c9dc3275565232e727dfe2dd796745080f55cff1b71cf430e30d8"
Sep 17 21:13:26.140792 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:26.140749    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf"]
Sep 17 21:13:26.143813 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:26.143795    2567 scope.go:117] "RemoveContainer" containerID="9218316b784653d82119d5a5e44b3e8cf0a2ca5d4810e80a94982a82d8d4fa23"
Sep 17 21:13:26.144107 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:13:26.144087    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9218316b784653d82119d5a5e44b3e8cf0a2ca5d4810e80a94982a82d8d4fa23\": container with ID starting with 9218316b784653d82119d5a5e44b3e8cf0a2ca5d4810e80a94982a82d8d4fa23 not found: ID does not exist" containerID="9218316b784653d82119d5a5e44b3e8cf0a2ca5d4810e80a94982a82d8d4fa23"
Sep 17 21:13:26.144194 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:26.144118    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9218316b784653d82119d5a5e44b3e8cf0a2ca5d4810e80a94982a82d8d4fa23"} err="failed to get container status \"9218316b784653d82119d5a5e44b3e8cf0a2ca5d4810e80a94982a82d8d4fa23\": rpc error: code = NotFound desc = could not find container \"9218316b784653d82119d5a5e44b3e8cf0a2ca5d4810e80a94982a82d8d4fa23\": container with ID starting with 9218316b784653d82119d5a5e44b3e8cf0a2ca5d4810e80a94982a82d8d4fa23 not found: ID does not exist"
Sep 17 21:13:26.144194 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:26.144141    2567 scope.go:117] "RemoveContainer" containerID="cb5ac1ed8696c6271131c404182fea203ef74470603706f740d0b6a0e6b2bf41"
Sep 17 21:13:26.144400 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:13:26.144376    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb5ac1ed8696c6271131c404182fea203ef74470603706f740d0b6a0e6b2bf41\": container with ID starting with cb5ac1ed8696c6271131c404182fea203ef74470603706f740d0b6a0e6b2bf41 not found: ID does not exist" containerID="cb5ac1ed8696c6271131c404182fea203ef74470603706f740d0b6a0e6b2bf41"
Sep 17 21:13:26.144564 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:26.144407    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb5ac1ed8696c6271131c404182fea203ef74470603706f740d0b6a0e6b2bf41"} err="failed to get container status \"cb5ac1ed8696c6271131c404182fea203ef74470603706f740d0b6a0e6b2bf41\": rpc error: code = NotFound desc = could not find container \"cb5ac1ed8696c6271131c404182fea203ef74470603706f740d0b6a0e6b2bf41\": container with ID starting with cb5ac1ed8696c6271131c404182fea203ef74470603706f740d0b6a0e6b2bf41 not found: ID does not exist"
Sep 17 21:13:26.144564 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:26.144427    2567 scope.go:117] "RemoveContainer" containerID="ac8afd113f7c9dc3275565232e727dfe2dd796745080f55cff1b71cf430e30d8"
Sep 17 21:13:26.144721 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:13:26.144700    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac8afd113f7c9dc3275565232e727dfe2dd796745080f55cff1b71cf430e30d8\": container with ID starting with ac8afd113f7c9dc3275565232e727dfe2dd796745080f55cff1b71cf430e30d8 not found: ID does not exist" containerID="ac8afd113f7c9dc3275565232e727dfe2dd796745080f55cff1b71cf430e30d8"
Sep 17 21:13:26.144786 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:26.144729    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac8afd113f7c9dc3275565232e727dfe2dd796745080f55cff1b71cf430e30d8"} err="failed to get container status \"ac8afd113f7c9dc3275565232e727dfe2dd796745080f55cff1b71cf430e30d8\": rpc error: code = NotFound desc = could not find container \"ac8afd113f7c9dc3275565232e727dfe2dd796745080f55cff1b71cf430e30d8\": container with ID starting with ac8afd113f7c9dc3275565232e727dfe2dd796745080f55cff1b71cf430e30d8 not found: ID does not exist"
Sep 17 21:13:26.147159 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:26.147121    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:13:26.147257 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:26.147165    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-create-v2-predictor-bfcdb48dd-jggsf"]
Sep 17 21:13:26.169053 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:26.169018    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e72d4ce2-ee89-4167-9628-05705846e4ca" path="/var/lib/kubelet/pods/e72d4ce2-ee89-4167-9628-05705846e4ca/volumes"
Sep 17 21:13:26.565627 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:26.565593    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ffb8954a-ca80-459c-974e-0d549edd1e6c-proxy-tls\") pod \"isvc-canary-promote-predictor-78b4c78fb9-8pzzt\" (UID: \"ffb8954a-ca80-459c-974e-0d549edd1e6c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt"
Sep 17 21:13:26.567836 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:26.567816    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ffb8954a-ca80-459c-974e-0d549edd1e6c-proxy-tls\") pod \"isvc-canary-promote-predictor-78b4c78fb9-8pzzt\" (UID: \"ffb8954a-ca80-459c-974e-0d549edd1e6c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt"
Sep 17 21:13:26.861224 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:26.861118    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt"
Sep 17 21:13:27.135747 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:27.135722    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt"]
Sep 17 21:13:27.137922 ip-10-0-132-201 kubenswrapper[2567]: W0917 21:13:27.137892    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffb8954a_ca80_459c_974e_0d549edd1e6c.slice/crio-a6f468be1265088e64b62c726001059122e954345119681f686bea08a8054a2b WatchSource:0}: Error finding container a6f468be1265088e64b62c726001059122e954345119681f686bea08a8054a2b: Status 404 returned error can't find the container with id a6f468be1265088e64b62c726001059122e954345119681f686bea08a8054a2b
Sep 17 21:13:28.128345 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:28.128306    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt" event={"ID":"ffb8954a-ca80-459c-974e-0d549edd1e6c","Type":"ContainerStarted","Data":"910b93880c423bacb80698bb9c9852748e08478d0c3870a13ab68c0b18469686"}
Sep 17 21:13:28.128748 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:28.128378    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt" event={"ID":"ffb8954a-ca80-459c-974e-0d549edd1e6c","Type":"ContainerStarted","Data":"a6f468be1265088e64b62c726001059122e954345119681f686bea08a8054a2b"}
Sep 17 21:13:31.142326 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:31.142290    2567 generic.go:356] "Generic (PLEG): container finished" podID="ffb8954a-ca80-459c-974e-0d549edd1e6c" containerID="910b93880c423bacb80698bb9c9852748e08478d0c3870a13ab68c0b18469686" exitCode=0
Sep 17 21:13:31.142797 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:31.142372    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt" event={"ID":"ffb8954a-ca80-459c-974e-0d549edd1e6c","Type":"ContainerDied","Data":"910b93880c423bacb80698bb9c9852748e08478d0c3870a13ab68c0b18469686"}
Sep 17 21:13:32.148583 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:32.148550    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt" event={"ID":"ffb8954a-ca80-459c-974e-0d549edd1e6c","Type":"ContainerStarted","Data":"c3d212158a3888c8f4578875ce21f3670f869a107feaab739988f6ecdaefd21a"}
Sep 17 21:13:32.148976 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:32.148613    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt" event={"ID":"ffb8954a-ca80-459c-974e-0d549edd1e6c","Type":"ContainerStarted","Data":"0c408f80b6869d60b52a98765b1640e6f61fc4fa04e018bcc1293a2c5d7b5bb4"}
Sep 17 21:13:32.148976 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:32.148792    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt"
Sep 17 21:13:32.174740 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:32.174685    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt" podStartSLOduration=3.314030922 podStartE2EDuration="7.17467097s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.860640048s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:13:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:13:32.172362462 +0000 UTC m=+1476.517964300" watchObservedRunningTime="2026-09-17 21:13:32.17467097 +0000 UTC m=+1476.520272794"
Sep 17 21:13:33.153141 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:33.153109    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt"
Sep 17 21:13:33.154351 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:33.154324    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt" podUID="ffb8954a-ca80-459c-974e-0d549edd1e6c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 17 21:13:34.157251 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:34.157205    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt" podUID="ffb8954a-ca80-459c-974e-0d549edd1e6c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 17 21:13:39.162220 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:39.162188    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt"
Sep 17 21:13:39.162690 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:39.162642    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt" podUID="ffb8954a-ca80-459c-974e-0d549edd1e6c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 17 21:13:49.163085 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:49.163048    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt" podUID="ffb8954a-ca80-459c-974e-0d549edd1e6c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 17 21:13:59.163270 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:13:59.163226    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt" podUID="ffb8954a-ca80-459c-974e-0d549edd1e6c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 17 21:14:09.163466 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:09.163423    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt" podUID="ffb8954a-ca80-459c-974e-0d549edd1e6c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 17 21:14:19.162960 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:19.162872    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt" podUID="ffb8954a-ca80-459c-974e-0d549edd1e6c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 17 21:14:29.163268 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:29.163222    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt" podUID="ffb8954a-ca80-459c-974e-0d549edd1e6c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 17 21:14:39.162771 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:39.162724    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt" podUID="ffb8954a-ca80-459c-974e-0d549edd1e6c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 17 21:14:41.164518 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:41.164474    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt" podUID="ffb8954a-ca80-459c-974e-0d549edd1e6c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 17 21:14:51.165920 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:51.165880    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt"
Sep 17 21:14:56.170274 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:56.170244    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt"]
Sep 17 21:14:56.170653 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:56.170495    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt" podUID="ffb8954a-ca80-459c-974e-0d549edd1e6c" containerName="kserve-container" containerID="cri-o://0c408f80b6869d60b52a98765b1640e6f61fc4fa04e018bcc1293a2c5d7b5bb4" gracePeriod=30
Sep 17 21:14:56.170653 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:56.170516    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt" podUID="ffb8954a-ca80-459c-974e-0d549edd1e6c" containerName="kube-rbac-proxy" containerID="cri-o://c3d212158a3888c8f4578875ce21f3670f869a107feaab739988f6ecdaefd21a" gracePeriod=30
Sep 17 21:14:56.246916 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:56.246881    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:14:56.269169 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:56.269135    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w"]
Sep 17 21:14:56.272795 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:56.272752    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w"
Sep 17 21:14:56.275673 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:56.275649    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-promote-v2-predictor-serving-cert\""
Sep 17 21:14:56.288145 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:56.288122    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w"]
Sep 17 21:14:56.391642 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:56.391606    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmxw9\" (UniqueName: \"kubernetes.io/projected/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93-kube-api-access-cmxw9\") pod \"isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w\" (UID: \"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w"
Sep 17 21:14:56.391816 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:56.391648    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w\" (UID: \"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w"
Sep 17 21:14:56.391816 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:56.391687    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w\" (UID: \"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w"
Sep 17 21:14:56.391816 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:56.391708    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w\" (UID: \"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w"
Sep 17 21:14:56.391816 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:56.391739    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w\" (UID: \"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w"
Sep 17 21:14:56.492160 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:56.492139    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmxw9\" (UniqueName: \"kubernetes.io/projected/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93-kube-api-access-cmxw9\") pod \"isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w\" (UID: \"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w"
Sep 17 21:14:56.492344 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:56.492169    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w\" (UID: \"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w"
Sep 17 21:14:56.492344 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:56.492204    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w\" (UID: \"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w"
Sep 17 21:14:56.492344 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:56.492228    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w\" (UID: \"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w"
Sep 17 21:14:56.492344 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:56.492273    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w\" (UID: \"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w"
Sep 17 21:14:56.492633 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:56.492592    2567 generic.go:356] "Generic (PLEG): container finished" podID="ffb8954a-ca80-459c-974e-0d549edd1e6c" containerID="c3d212158a3888c8f4578875ce21f3670f869a107feaab739988f6ecdaefd21a" exitCode=2
Sep 17 21:14:56.492633 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:56.492623    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt" event={"ID":"ffb8954a-ca80-459c-974e-0d549edd1e6c","Type":"ContainerDied","Data":"c3d212158a3888c8f4578875ce21f3670f869a107feaab739988f6ecdaefd21a"}
Sep 17 21:14:56.492863 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:56.492601    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w\" (UID: \"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w"
Sep 17 21:14:56.492995 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:56.492979    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w\" (UID: \"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w"
Sep 17 21:14:56.493049 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:56.492996    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w\" (UID: \"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w"
Sep 17 21:14:56.494892 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:56.494873    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w\" (UID: \"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w"
Sep 17 21:14:56.501740 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:56.501715    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmxw9\" (UniqueName: \"kubernetes.io/projected/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93-kube-api-access-cmxw9\") pod \"isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w\" (UID: \"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w"
Sep 17 21:14:56.583668 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:56.583640    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w"
Sep 17 21:14:56.863009 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:56.862935    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w"]
Sep 17 21:14:56.865518 ip-10-0-132-201 kubenswrapper[2567]: W0917 21:14:56.865493    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40aa0f85_2c9d_4ac2_9eb2_d42e9f24ae93.slice/crio-17c594fa417d11ef462e40bf03a2417d1302e61702f22c0966ba408ac12b191f WatchSource:0}: Error finding container 17c594fa417d11ef462e40bf03a2417d1302e61702f22c0966ba408ac12b191f: Status 404 returned error can't find the container with id 17c594fa417d11ef462e40bf03a2417d1302e61702f22c0966ba408ac12b191f
Sep 17 21:14:57.500911 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:57.500873    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w" event={"ID":"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93","Type":"ContainerStarted","Data":"d8eb4feb63d6b18674074188b54d5e16b746ef308f30b60f318e4d1647f3c894"}
Sep 17 21:14:57.501383 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:57.500943    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w" event={"ID":"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93","Type":"ContainerStarted","Data":"17c594fa417d11ef462e40bf03a2417d1302e61702f22c0966ba408ac12b191f"}
Sep 17 21:14:57.915007 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:57.914913    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:14:59.157957 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:14:59.157912    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt" podUID="ffb8954a-ca80-459c-974e-0d549edd1e6c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.34:8643/healthz\": dial tcp 10.132.0.34:8643: connect: connection refused"
Sep 17 21:15:01.213428 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.213405    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt"
Sep 17 21:15:01.332719 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.332687    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ffb8954a-ca80-459c-974e-0d549edd1e6c-kube-rbac-proxy-sar-config\") pod \"ffb8954a-ca80-459c-974e-0d549edd1e6c\" (UID: \"ffb8954a-ca80-459c-974e-0d549edd1e6c\") "
Sep 17 21:15:01.332891 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.332728    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ffb8954a-ca80-459c-974e-0d549edd1e6c-cabundle-cert\") pod \"ffb8954a-ca80-459c-974e-0d549edd1e6c\" (UID: \"ffb8954a-ca80-459c-974e-0d549edd1e6c\") "
Sep 17 21:15:01.332891 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.332823    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p62bh\" (UniqueName: \"kubernetes.io/projected/ffb8954a-ca80-459c-974e-0d549edd1e6c-kube-api-access-p62bh\") pod \"ffb8954a-ca80-459c-974e-0d549edd1e6c\" (UID: \"ffb8954a-ca80-459c-974e-0d549edd1e6c\") "
Sep 17 21:15:01.332891 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.332871    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ffb8954a-ca80-459c-974e-0d549edd1e6c-kserve-provision-location\") pod \"ffb8954a-ca80-459c-974e-0d549edd1e6c\" (UID: \"ffb8954a-ca80-459c-974e-0d549edd1e6c\") "
Sep 17 21:15:01.333005 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.332969    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ffb8954a-ca80-459c-974e-0d549edd1e6c-proxy-tls\") pod \"ffb8954a-ca80-459c-974e-0d549edd1e6c\" (UID: \"ffb8954a-ca80-459c-974e-0d549edd1e6c\") "
Sep 17 21:15:01.333233 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.333205    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffb8954a-ca80-459c-974e-0d549edd1e6c-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "ffb8954a-ca80-459c-974e-0d549edd1e6c" (UID: "ffb8954a-ca80-459c-974e-0d549edd1e6c"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 21:15:01.333233 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.333226    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffb8954a-ca80-459c-974e-0d549edd1e6c-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "ffb8954a-ca80-459c-974e-0d549edd1e6c" (UID: "ffb8954a-ca80-459c-974e-0d549edd1e6c"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:15:01.333370 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.333209    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffb8954a-ca80-459c-974e-0d549edd1e6c-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "ffb8954a-ca80-459c-974e-0d549edd1e6c" (UID: "ffb8954a-ca80-459c-974e-0d549edd1e6c"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:15:01.334987 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.334956    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffb8954a-ca80-459c-974e-0d549edd1e6c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ffb8954a-ca80-459c-974e-0d549edd1e6c" (UID: "ffb8954a-ca80-459c-974e-0d549edd1e6c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:15:01.335094 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.335059    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffb8954a-ca80-459c-974e-0d549edd1e6c-kube-api-access-p62bh" (OuterVolumeSpecName: "kube-api-access-p62bh") pod "ffb8954a-ca80-459c-974e-0d549edd1e6c" (UID: "ffb8954a-ca80-459c-974e-0d549edd1e6c"). InnerVolumeSpecName "kube-api-access-p62bh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:15:01.433999 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.433909    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ffb8954a-ca80-459c-974e-0d549edd1e6c-proxy-tls\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:15:01.433999 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.433942    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ffb8954a-ca80-459c-974e-0d549edd1e6c-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:15:01.433999 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.433952    2567 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ffb8954a-ca80-459c-974e-0d549edd1e6c-cabundle-cert\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:15:01.433999 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.433961    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p62bh\" (UniqueName: \"kubernetes.io/projected/ffb8954a-ca80-459c-974e-0d549edd1e6c-kube-api-access-p62bh\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:15:01.433999 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.433969    2567 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ffb8954a-ca80-459c-974e-0d549edd1e6c-kserve-provision-location\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:15:01.520052 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.520017    2567 generic.go:356] "Generic (PLEG): container finished" podID="ffb8954a-ca80-459c-974e-0d549edd1e6c" containerID="0c408f80b6869d60b52a98765b1640e6f61fc4fa04e018bcc1293a2c5d7b5bb4" exitCode=0
Sep 17 21:15:01.520217 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.520077    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt"
Sep 17 21:15:01.520217 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.520109    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt" event={"ID":"ffb8954a-ca80-459c-974e-0d549edd1e6c","Type":"ContainerDied","Data":"0c408f80b6869d60b52a98765b1640e6f61fc4fa04e018bcc1293a2c5d7b5bb4"}
Sep 17 21:15:01.520217 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.520165    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt" event={"ID":"ffb8954a-ca80-459c-974e-0d549edd1e6c","Type":"ContainerDied","Data":"a6f468be1265088e64b62c726001059122e954345119681f686bea08a8054a2b"}
Sep 17 21:15:01.520217 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.520189    2567 scope.go:117] "RemoveContainer" containerID="c3d212158a3888c8f4578875ce21f3670f869a107feaab739988f6ecdaefd21a"
Sep 17 21:15:01.521793 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.521752    2567 generic.go:356] "Generic (PLEG): container finished" podID="40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" containerID="d8eb4feb63d6b18674074188b54d5e16b746ef308f30b60f318e4d1647f3c894" exitCode=0
Sep 17 21:15:01.521793 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.521786    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w" event={"ID":"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93","Type":"ContainerDied","Data":"d8eb4feb63d6b18674074188b54d5e16b746ef308f30b60f318e4d1647f3c894"}
Sep 17 21:15:01.529129 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.529018    2567 scope.go:117] "RemoveContainer" containerID="0c408f80b6869d60b52a98765b1640e6f61fc4fa04e018bcc1293a2c5d7b5bb4"
Sep 17 21:15:01.536499 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.536475    2567 scope.go:117] "RemoveContainer" containerID="910b93880c423bacb80698bb9c9852748e08478d0c3870a13ab68c0b18469686"
Sep 17 21:15:01.544904 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.544882    2567 scope.go:117] "RemoveContainer" containerID="c3d212158a3888c8f4578875ce21f3670f869a107feaab739988f6ecdaefd21a"
Sep 17 21:15:01.545214 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:15:01.545196    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3d212158a3888c8f4578875ce21f3670f869a107feaab739988f6ecdaefd21a\": container with ID starting with c3d212158a3888c8f4578875ce21f3670f869a107feaab739988f6ecdaefd21a not found: ID does not exist" containerID="c3d212158a3888c8f4578875ce21f3670f869a107feaab739988f6ecdaefd21a"
Sep 17 21:15:01.545258 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.545225    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3d212158a3888c8f4578875ce21f3670f869a107feaab739988f6ecdaefd21a"} err="failed to get container status \"c3d212158a3888c8f4578875ce21f3670f869a107feaab739988f6ecdaefd21a\": rpc error: code = NotFound desc = could not find container \"c3d212158a3888c8f4578875ce21f3670f869a107feaab739988f6ecdaefd21a\": container with ID starting with c3d212158a3888c8f4578875ce21f3670f869a107feaab739988f6ecdaefd21a not found: ID does not exist"
Sep 17 21:15:01.545258 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.545241    2567 scope.go:117] "RemoveContainer" containerID="0c408f80b6869d60b52a98765b1640e6f61fc4fa04e018bcc1293a2c5d7b5bb4"
Sep 17 21:15:01.545490 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:15:01.545472    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c408f80b6869d60b52a98765b1640e6f61fc4fa04e018bcc1293a2c5d7b5bb4\": container with ID starting with 0c408f80b6869d60b52a98765b1640e6f61fc4fa04e018bcc1293a2c5d7b5bb4 not found: ID does not exist" containerID="0c408f80b6869d60b52a98765b1640e6f61fc4fa04e018bcc1293a2c5d7b5bb4"
Sep 17 21:15:01.545535 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.545497    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c408f80b6869d60b52a98765b1640e6f61fc4fa04e018bcc1293a2c5d7b5bb4"} err="failed to get container status \"0c408f80b6869d60b52a98765b1640e6f61fc4fa04e018bcc1293a2c5d7b5bb4\": rpc error: code = NotFound desc = could not find container \"0c408f80b6869d60b52a98765b1640e6f61fc4fa04e018bcc1293a2c5d7b5bb4\": container with ID starting with 0c408f80b6869d60b52a98765b1640e6f61fc4fa04e018bcc1293a2c5d7b5bb4 not found: ID does not exist"
Sep 17 21:15:01.545535 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.545512    2567 scope.go:117] "RemoveContainer" containerID="910b93880c423bacb80698bb9c9852748e08478d0c3870a13ab68c0b18469686"
Sep 17 21:15:01.545790 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:15:01.545748    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"910b93880c423bacb80698bb9c9852748e08478d0c3870a13ab68c0b18469686\": container with ID starting with 910b93880c423bacb80698bb9c9852748e08478d0c3870a13ab68c0b18469686 not found: ID does not exist" containerID="910b93880c423bacb80698bb9c9852748e08478d0c3870a13ab68c0b18469686"
Sep 17 21:15:01.545876 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.545795    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"910b93880c423bacb80698bb9c9852748e08478d0c3870a13ab68c0b18469686"} err="failed to get container status \"910b93880c423bacb80698bb9c9852748e08478d0c3870a13ab68c0b18469686\": rpc error: code = NotFound desc = could not find container \"910b93880c423bacb80698bb9c9852748e08478d0c3870a13ab68c0b18469686\": container with ID starting with 910b93880c423bacb80698bb9c9852748e08478d0c3870a13ab68c0b18469686 not found: ID does not exist"
Sep 17 21:15:01.582169 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.582142    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt"]
Sep 17 21:15:01.595740 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:01.595715    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt"]
Sep 17 21:15:02.165557 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:02.165511    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-78b4c78fb9-8pzzt" podUID="ffb8954a-ca80-459c-974e-0d549edd1e6c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: i/o timeout"
Sep 17 21:15:02.169591 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:02.169560    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffb8954a-ca80-459c-974e-0d549edd1e6c" path="/var/lib/kubelet/pods/ffb8954a-ca80-459c-974e-0d549edd1e6c/volumes"
Sep 17 21:15:02.529319 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:02.529284    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w" event={"ID":"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93","Type":"ContainerStarted","Data":"ee264f5bd0ba93f5c8deeffc3f0b88fceb3469e6d7f36722da0eb271c8649f59"}
Sep 17 21:15:02.529836 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:02.529353    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w" event={"ID":"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93","Type":"ContainerStarted","Data":"fcfd96b39aad56649210fcd2a41308a2684f808aef6d2859732d50be6b394bca"}
Sep 17 21:15:02.529836 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:02.529567    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w"
Sep 17 21:15:02.554574 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:02.554529    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w" podStartSLOduration=2.6119563550000002 podStartE2EDuration="6.554515855s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.9425595s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:14:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:15:02.553253933 +0000 UTC m=+1566.898855815" watchObservedRunningTime="2026-09-17 21:15:02.554515855 +0000 UTC m=+1566.900117679"
Sep 17 21:15:03.533181 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:03.533144    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w"
Sep 17 21:15:03.534557 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:03.534528    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w" podUID="40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Sep 17 21:15:04.537741 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:04.537697    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w" podUID="40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Sep 17 21:15:09.541696 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:09.541667    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w"
Sep 17 21:15:09.542263 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:09.542235    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w" podUID="40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Sep 17 21:15:19.542560 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:19.542513    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w" podUID="40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Sep 17 21:15:29.542630 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:29.542586    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w" podUID="40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Sep 17 21:15:39.542186 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:39.542129    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w" podUID="40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Sep 17 21:15:49.542732 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:49.542643    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w" podUID="40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Sep 17 21:15:59.542701 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:15:59.542660    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w" podUID="40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Sep 17 21:16:09.542302 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:09.542258    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w" podUID="40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Sep 17 21:16:19.542571 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:19.542530    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w" podUID="40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Sep 17 21:16:29.165699 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:29.165668    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w"
Sep 17 21:16:36.380987 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:36.380945    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w"]
Sep 17 21:16:36.381449 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:36.381251    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w" podUID="40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" containerName="kserve-container" containerID="cri-o://fcfd96b39aad56649210fcd2a41308a2684f808aef6d2859732d50be6b394bca" gracePeriod=30
Sep 17 21:16:36.381449 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:36.381328    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w" podUID="40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" containerName="kube-rbac-proxy" containerID="cri-o://ee264f5bd0ba93f5c8deeffc3f0b88fceb3469e6d7f36722da0eb271c8649f59" gracePeriod=30
Sep 17 21:16:36.912837 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:36.912803    2567 generic.go:356] "Generic (PLEG): container finished" podID="40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" containerID="ee264f5bd0ba93f5c8deeffc3f0b88fceb3469e6d7f36722da0eb271c8649f59" exitCode=2
Sep 17 21:16:36.913037 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:36.912883    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w" event={"ID":"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93","Type":"ContainerDied","Data":"ee264f5bd0ba93f5c8deeffc3f0b88fceb3469e6d7f36722da0eb271c8649f59"}
Sep 17 21:16:37.567727 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:37.567691    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:16:39.165823 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:39.165779    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w" podUID="40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Sep 17 21:16:39.538713 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:39.538663    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w" podUID="40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.35:8643/healthz\": dial tcp 10.132.0.35:8643: connect: connection refused"
Sep 17 21:16:41.429982 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.429953    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w"
Sep 17 21:16:41.462158 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.462124    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93-cabundle-cert\") pod \"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93\" (UID: \"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93\") "
Sep 17 21:16:41.462329 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.462167    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cmxw9\" (UniqueName: \"kubernetes.io/projected/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93-kube-api-access-cmxw9\") pod \"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93\" (UID: \"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93\") "
Sep 17 21:16:41.462329 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.462231    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93-kserve-provision-location\") pod \"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93\" (UID: \"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93\") "
Sep 17 21:16:41.462329 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.462251    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93-kube-rbac-proxy-sar-config\") pod \"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93\" (UID: \"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93\") "
Sep 17 21:16:41.462329 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.462270    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93-proxy-tls\") pod \"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93\" (UID: \"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93\") "
Sep 17 21:16:41.462598 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.462571    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" (UID: "40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:16:41.462598 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.462578    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" (UID: "40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 21:16:41.462688 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.462641    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" (UID: "40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:16:41.464354 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.464331    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93-kube-api-access-cmxw9" (OuterVolumeSpecName: "kube-api-access-cmxw9") pod "40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" (UID: "40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93"). InnerVolumeSpecName "kube-api-access-cmxw9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:16:41.464513 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.464485    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" (UID: "40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:16:41.563203 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.563132    2567 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93-cabundle-cert\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:16:41.563203 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.563157    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmxw9\" (UniqueName: \"kubernetes.io/projected/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93-kube-api-access-cmxw9\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:16:41.563203 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.563166    2567 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93-kserve-provision-location\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:16:41.563203 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.563175    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:16:41.563203 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.563184    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93-proxy-tls\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:16:41.934311 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.934222    2567 generic.go:356] "Generic (PLEG): container finished" podID="40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" containerID="fcfd96b39aad56649210fcd2a41308a2684f808aef6d2859732d50be6b394bca" exitCode=0
Sep 17 21:16:41.934311 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.934286    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w"
Sep 17 21:16:41.934493 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.934306    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w" event={"ID":"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93","Type":"ContainerDied","Data":"fcfd96b39aad56649210fcd2a41308a2684f808aef6d2859732d50be6b394bca"}
Sep 17 21:16:41.934493 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.934365    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w" event={"ID":"40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93","Type":"ContainerDied","Data":"17c594fa417d11ef462e40bf03a2417d1302e61702f22c0966ba408ac12b191f"}
Sep 17 21:16:41.934493 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.934392    2567 scope.go:117] "RemoveContainer" containerID="ee264f5bd0ba93f5c8deeffc3f0b88fceb3469e6d7f36722da0eb271c8649f59"
Sep 17 21:16:41.945156 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.943608    2567 scope.go:117] "RemoveContainer" containerID="fcfd96b39aad56649210fcd2a41308a2684f808aef6d2859732d50be6b394bca"
Sep 17 21:16:41.951379 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.951359    2567 scope.go:117] "RemoveContainer" containerID="d8eb4feb63d6b18674074188b54d5e16b746ef308f30b60f318e4d1647f3c894"
Sep 17 21:16:41.956805 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.956781    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w"]
Sep 17 21:16:41.959536 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.959518    2567 scope.go:117] "RemoveContainer" containerID="ee264f5bd0ba93f5c8deeffc3f0b88fceb3469e6d7f36722da0eb271c8649f59"
Sep 17 21:16:41.959829 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:16:41.959809    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee264f5bd0ba93f5c8deeffc3f0b88fceb3469e6d7f36722da0eb271c8649f59\": container with ID starting with ee264f5bd0ba93f5c8deeffc3f0b88fceb3469e6d7f36722da0eb271c8649f59 not found: ID does not exist" containerID="ee264f5bd0ba93f5c8deeffc3f0b88fceb3469e6d7f36722da0eb271c8649f59"
Sep 17 21:16:41.959889 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.959836    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee264f5bd0ba93f5c8deeffc3f0b88fceb3469e6d7f36722da0eb271c8649f59"} err="failed to get container status \"ee264f5bd0ba93f5c8deeffc3f0b88fceb3469e6d7f36722da0eb271c8649f59\": rpc error: code = NotFound desc = could not find container \"ee264f5bd0ba93f5c8deeffc3f0b88fceb3469e6d7f36722da0eb271c8649f59\": container with ID starting with ee264f5bd0ba93f5c8deeffc3f0b88fceb3469e6d7f36722da0eb271c8649f59 not found: ID does not exist"
Sep 17 21:16:41.959889 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.959855    2567 scope.go:117] "RemoveContainer" containerID="fcfd96b39aad56649210fcd2a41308a2684f808aef6d2859732d50be6b394bca"
Sep 17 21:16:41.960122 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:16:41.960104    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcfd96b39aad56649210fcd2a41308a2684f808aef6d2859732d50be6b394bca\": container with ID starting with fcfd96b39aad56649210fcd2a41308a2684f808aef6d2859732d50be6b394bca not found: ID does not exist" containerID="fcfd96b39aad56649210fcd2a41308a2684f808aef6d2859732d50be6b394bca"
Sep 17 21:16:41.960163 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.960129    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcfd96b39aad56649210fcd2a41308a2684f808aef6d2859732d50be6b394bca"} err="failed to get container status \"fcfd96b39aad56649210fcd2a41308a2684f808aef6d2859732d50be6b394bca\": rpc error: code = NotFound desc = could not find container \"fcfd96b39aad56649210fcd2a41308a2684f808aef6d2859732d50be6b394bca\": container with ID starting with fcfd96b39aad56649210fcd2a41308a2684f808aef6d2859732d50be6b394bca not found: ID does not exist"
Sep 17 21:16:41.960163 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.960145    2567 scope.go:117] "RemoveContainer" containerID="d8eb4feb63d6b18674074188b54d5e16b746ef308f30b60f318e4d1647f3c894"
Sep 17 21:16:41.960392 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:16:41.960373    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8eb4feb63d6b18674074188b54d5e16b746ef308f30b60f318e4d1647f3c894\": container with ID starting with d8eb4feb63d6b18674074188b54d5e16b746ef308f30b60f318e4d1647f3c894 not found: ID does not exist" containerID="d8eb4feb63d6b18674074188b54d5e16b746ef308f30b60f318e4d1647f3c894"
Sep 17 21:16:41.960432 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.960398    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8eb4feb63d6b18674074188b54d5e16b746ef308f30b60f318e4d1647f3c894"} err="failed to get container status \"d8eb4feb63d6b18674074188b54d5e16b746ef308f30b60f318e4d1647f3c894\": rpc error: code = NotFound desc = could not find container \"d8eb4feb63d6b18674074188b54d5e16b746ef308f30b60f318e4d1647f3c894\": container with ID starting with d8eb4feb63d6b18674074188b54d5e16b746ef308f30b60f318e4d1647f3c894 not found: ID does not exist"
Sep 17 21:16:41.960845 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:41.960829    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-79d9bd5fdb-wbc5w"]
Sep 17 21:16:42.169404 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:42.169371    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" path="/var/lib/kubelet/pods/40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93/volumes"
Sep 17 21:16:51.064420 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.064388    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:16:51.105584 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.105547    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n"]
Sep 17 21:16:51.105908 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.105893    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" containerName="storage-initializer"
Sep 17 21:16:51.105953 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.105910    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" containerName="storage-initializer"
Sep 17 21:16:51.105953 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.105927    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffb8954a-ca80-459c-974e-0d549edd1e6c" containerName="kube-rbac-proxy"
Sep 17 21:16:51.105953 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.105933    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffb8954a-ca80-459c-974e-0d549edd1e6c" containerName="kube-rbac-proxy"
Sep 17 21:16:51.105953 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.105942    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" containerName="kube-rbac-proxy"
Sep 17 21:16:51.105953 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.105948    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" containerName="kube-rbac-proxy"
Sep 17 21:16:51.106108 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.105958    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffb8954a-ca80-459c-974e-0d549edd1e6c" containerName="kserve-container"
Sep 17 21:16:51.106108 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.105964    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffb8954a-ca80-459c-974e-0d549edd1e6c" containerName="kserve-container"
Sep 17 21:16:51.106108 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.105970    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" containerName="kserve-container"
Sep 17 21:16:51.106108 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.105976    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" containerName="kserve-container"
Sep 17 21:16:51.106108 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.105983    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffb8954a-ca80-459c-974e-0d549edd1e6c" containerName="storage-initializer"
Sep 17 21:16:51.106108 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.105988    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffb8954a-ca80-459c-974e-0d549edd1e6c" containerName="storage-initializer"
Sep 17 21:16:51.106108 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.106032    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffb8954a-ca80-459c-974e-0d549edd1e6c" containerName="kserve-container"
Sep 17 21:16:51.106108 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.106043    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" containerName="kserve-container"
Sep 17 21:16:51.106108 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.106050    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="40aa0f85-2c9d-4ac2-9eb2-d42e9f24ae93" containerName="kube-rbac-proxy"
Sep 17 21:16:51.106108 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.106055    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffb8954a-ca80-459c-974e-0d549edd1e6c" containerName="kube-rbac-proxy"
Sep 17 21:16:51.110595 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.110575    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n"
Sep 17 21:16:51.113530 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.113508    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 17 21:16:51.113648 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.113538    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 17 21:16:51.113779 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.113744    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-5dj6n\""
Sep 17 21:16:51.113847 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.113747    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"message-dumper-raw-cipn-kube-rbac-proxy-sar-config\""
Sep 17 21:16:51.113983 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.113970    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"message-dumper-raw-cipn-predictor-serving-cert\""
Sep 17 21:16:51.119146 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.119123    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n"]
Sep 17 21:16:51.143593 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.143561    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dd99d6bd-8a68-4afb-8254-bf595ae781a3-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-cipn-predictor-9774bf55d-8gg4n\" (UID: \"dd99d6bd-8a68-4afb-8254-bf595ae781a3\") " pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n"
Sep 17 21:16:51.143750 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.143613    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shgfh\" (UniqueName: \"kubernetes.io/projected/dd99d6bd-8a68-4afb-8254-bf595ae781a3-kube-api-access-shgfh\") pod \"message-dumper-raw-cipn-predictor-9774bf55d-8gg4n\" (UID: \"dd99d6bd-8a68-4afb-8254-bf595ae781a3\") " pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n"
Sep 17 21:16:51.143750 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.143634    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd99d6bd-8a68-4afb-8254-bf595ae781a3-proxy-tls\") pod \"message-dumper-raw-cipn-predictor-9774bf55d-8gg4n\" (UID: \"dd99d6bd-8a68-4afb-8254-bf595ae781a3\") " pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n"
Sep 17 21:16:51.244016 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.243980    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shgfh\" (UniqueName: \"kubernetes.io/projected/dd99d6bd-8a68-4afb-8254-bf595ae781a3-kube-api-access-shgfh\") pod \"message-dumper-raw-cipn-predictor-9774bf55d-8gg4n\" (UID: \"dd99d6bd-8a68-4afb-8254-bf595ae781a3\") " pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n"
Sep 17 21:16:51.244016 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.244016    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd99d6bd-8a68-4afb-8254-bf595ae781a3-proxy-tls\") pod \"message-dumper-raw-cipn-predictor-9774bf55d-8gg4n\" (UID: \"dd99d6bd-8a68-4afb-8254-bf595ae781a3\") " pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n"
Sep 17 21:16:51.244229 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:16:51.244136    2567 secret.go:189] Couldn't get secret kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-serving-cert: secret "message-dumper-raw-cipn-predictor-serving-cert" not found
Sep 17 21:16:51.244229 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:16:51.244183    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd99d6bd-8a68-4afb-8254-bf595ae781a3-proxy-tls podName:dd99d6bd-8a68-4afb-8254-bf595ae781a3 nodeName:}" failed. No retries permitted until 2026-09-17 21:16:51.744169088 +0000 UTC m=+1676.089770892 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/dd99d6bd-8a68-4afb-8254-bf595ae781a3-proxy-tls") pod "message-dumper-raw-cipn-predictor-9774bf55d-8gg4n" (UID: "dd99d6bd-8a68-4afb-8254-bf595ae781a3") : secret "message-dumper-raw-cipn-predictor-serving-cert" not found
Sep 17 21:16:51.244229 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.244198    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dd99d6bd-8a68-4afb-8254-bf595ae781a3-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-cipn-predictor-9774bf55d-8gg4n\" (UID: \"dd99d6bd-8a68-4afb-8254-bf595ae781a3\") " pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n"
Sep 17 21:16:51.244908 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.244890    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dd99d6bd-8a68-4afb-8254-bf595ae781a3-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-cipn-predictor-9774bf55d-8gg4n\" (UID: \"dd99d6bd-8a68-4afb-8254-bf595ae781a3\") " pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n"
Sep 17 21:16:51.253168 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.253144    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shgfh\" (UniqueName: \"kubernetes.io/projected/dd99d6bd-8a68-4afb-8254-bf595ae781a3-kube-api-access-shgfh\") pod \"message-dumper-raw-cipn-predictor-9774bf55d-8gg4n\" (UID: \"dd99d6bd-8a68-4afb-8254-bf595ae781a3\") " pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n"
Sep 17 21:16:51.410884 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.410786    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:16:51.748218 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.748179    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd99d6bd-8a68-4afb-8254-bf595ae781a3-proxy-tls\") pod \"message-dumper-raw-cipn-predictor-9774bf55d-8gg4n\" (UID: \"dd99d6bd-8a68-4afb-8254-bf595ae781a3\") " pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n"
Sep 17 21:16:51.750576 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:51.750545    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd99d6bd-8a68-4afb-8254-bf595ae781a3-proxy-tls\") pod \"message-dumper-raw-cipn-predictor-9774bf55d-8gg4n\" (UID: \"dd99d6bd-8a68-4afb-8254-bf595ae781a3\") " pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n"
Sep 17 21:16:52.022096 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:52.022009    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n"
Sep 17 21:16:52.295733 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:52.295704    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n"]
Sep 17 21:16:52.297737 ip-10-0-132-201 kubenswrapper[2567]: W0917 21:16:52.297710    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd99d6bd_8a68_4afb_8254_bf595ae781a3.slice/crio-4df98032a759d0bda760b581e5e07d9632925b635802b92a7de93a418969ec39 WatchSource:0}: Error finding container 4df98032a759d0bda760b581e5e07d9632925b635802b92a7de93a418969ec39: Status 404 returned error can't find the container with id 4df98032a759d0bda760b581e5e07d9632925b635802b92a7de93a418969ec39
Sep 17 21:16:52.987374 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:52.987336    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n" event={"ID":"dd99d6bd-8a68-4afb-8254-bf595ae781a3","Type":"ContainerStarted","Data":"c6b03eb28aa953e5e248d21a351d6cabb5f0ab04a3eadfed9d7aa6a7403f1dd0"}
Sep 17 21:16:52.987551 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:52.987403    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n"
Sep 17 21:16:52.987551 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:52.987428    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n" event={"ID":"dd99d6bd-8a68-4afb-8254-bf595ae781a3","Type":"ContainerStarted","Data":"1a6145528b4bf7740a7efa61ace66e4886c65fe13b6240afcd869993d394b9cc"}
Sep 17 21:16:52.987551 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:52.987457    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n"
Sep 17 21:16:52.987551 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:52.987483    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n" event={"ID":"dd99d6bd-8a68-4afb-8254-bf595ae781a3","Type":"ContainerStarted","Data":"4df98032a759d0bda760b581e5e07d9632925b635802b92a7de93a418969ec39"}
Sep 17 21:16:52.989282 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:52.989259    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n"
Sep 17 21:16:53.006470 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:16:53.006423    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n" podStartSLOduration=2.006409645 podStartE2EDuration="2.006409645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:16:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:16:53.005997644 +0000 UTC m=+1677.351599489" watchObservedRunningTime="2026-09-17 21:16:53.006409645 +0000 UTC m=+1677.352011470"
Sep 17 21:17:00.000586 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:17:00.000554    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n"
Sep 17 21:17:01.101399 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:17:01.101363    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:17:01.293397 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:17:01.293366    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:18:36.662984 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:36.662949    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_message-dumper-raw-cipn-predictor-9774bf55d-8gg4n_dd99d6bd-8a68-4afb-8254-bf595ae781a3/kserve-container/0.log"
Sep 17 21:18:36.998255 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:36.998218    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n"]
Sep 17 21:18:36.998558 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:36.998531    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n" podUID="dd99d6bd-8a68-4afb-8254-bf595ae781a3" containerName="kserve-container" containerID="cri-o://1a6145528b4bf7740a7efa61ace66e4886c65fe13b6240afcd869993d394b9cc" gracePeriod=30
Sep 17 21:18:36.998657 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:36.998555    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n" podUID="dd99d6bd-8a68-4afb-8254-bf595ae781a3" containerName="kube-rbac-proxy" containerID="cri-o://c6b03eb28aa953e5e248d21a351d6cabb5f0ab04a3eadfed9d7aa6a7403f1dd0" gracePeriod=30
Sep 17 21:18:37.000620 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.000590    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:18:37.247501 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.247471    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n"
Sep 17 21:18:37.326669 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.326576    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shgfh\" (UniqueName: \"kubernetes.io/projected/dd99d6bd-8a68-4afb-8254-bf595ae781a3-kube-api-access-shgfh\") pod \"dd99d6bd-8a68-4afb-8254-bf595ae781a3\" (UID: \"dd99d6bd-8a68-4afb-8254-bf595ae781a3\") "
Sep 17 21:18:37.326669 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.326631    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dd99d6bd-8a68-4afb-8254-bf595ae781a3-kube-rbac-proxy-sar-config\") pod \"dd99d6bd-8a68-4afb-8254-bf595ae781a3\" (UID: \"dd99d6bd-8a68-4afb-8254-bf595ae781a3\") "
Sep 17 21:18:37.326936 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.326695    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd99d6bd-8a68-4afb-8254-bf595ae781a3-proxy-tls\") pod \"dd99d6bd-8a68-4afb-8254-bf595ae781a3\" (UID: \"dd99d6bd-8a68-4afb-8254-bf595ae781a3\") "
Sep 17 21:18:37.327122 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.327090    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd99d6bd-8a68-4afb-8254-bf595ae781a3-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "dd99d6bd-8a68-4afb-8254-bf595ae781a3" (UID: "dd99d6bd-8a68-4afb-8254-bf595ae781a3"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:18:37.328824 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.328794    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd99d6bd-8a68-4afb-8254-bf595ae781a3-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "dd99d6bd-8a68-4afb-8254-bf595ae781a3" (UID: "dd99d6bd-8a68-4afb-8254-bf595ae781a3"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:18:37.328924 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.328802    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd99d6bd-8a68-4afb-8254-bf595ae781a3-kube-api-access-shgfh" (OuterVolumeSpecName: "kube-api-access-shgfh") pod "dd99d6bd-8a68-4afb-8254-bf595ae781a3" (UID: "dd99d6bd-8a68-4afb-8254-bf595ae781a3"). InnerVolumeSpecName "kube-api-access-shgfh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:18:37.411828 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.411795    2567 generic.go:356] "Generic (PLEG): container finished" podID="dd99d6bd-8a68-4afb-8254-bf595ae781a3" containerID="c6b03eb28aa953e5e248d21a351d6cabb5f0ab04a3eadfed9d7aa6a7403f1dd0" exitCode=2
Sep 17 21:18:37.411828 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.411821    2567 generic.go:356] "Generic (PLEG): container finished" podID="dd99d6bd-8a68-4afb-8254-bf595ae781a3" containerID="1a6145528b4bf7740a7efa61ace66e4886c65fe13b6240afcd869993d394b9cc" exitCode=2
Sep 17 21:18:37.412064 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.411840    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n"
Sep 17 21:18:37.412064 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.411875    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n" event={"ID":"dd99d6bd-8a68-4afb-8254-bf595ae781a3","Type":"ContainerDied","Data":"c6b03eb28aa953e5e248d21a351d6cabb5f0ab04a3eadfed9d7aa6a7403f1dd0"}
Sep 17 21:18:37.412064 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.411948    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n" event={"ID":"dd99d6bd-8a68-4afb-8254-bf595ae781a3","Type":"ContainerDied","Data":"1a6145528b4bf7740a7efa61ace66e4886c65fe13b6240afcd869993d394b9cc"}
Sep 17 21:18:37.412064 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.411980    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n" event={"ID":"dd99d6bd-8a68-4afb-8254-bf595ae781a3","Type":"ContainerDied","Data":"4df98032a759d0bda760b581e5e07d9632925b635802b92a7de93a418969ec39"}
Sep 17 21:18:37.412064 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.412014    2567 scope.go:117] "RemoveContainer" containerID="c6b03eb28aa953e5e248d21a351d6cabb5f0ab04a3eadfed9d7aa6a7403f1dd0"
Sep 17 21:18:37.420117 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.420099    2567 scope.go:117] "RemoveContainer" containerID="1a6145528b4bf7740a7efa61ace66e4886c65fe13b6240afcd869993d394b9cc"
Sep 17 21:18:37.427549 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.427523    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shgfh\" (UniqueName: \"kubernetes.io/projected/dd99d6bd-8a68-4afb-8254-bf595ae781a3-kube-api-access-shgfh\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:18:37.427549 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.427545    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dd99d6bd-8a68-4afb-8254-bf595ae781a3-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:18:37.427707 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.427557    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd99d6bd-8a68-4afb-8254-bf595ae781a3-proxy-tls\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:18:37.427868 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.427704    2567 scope.go:117] "RemoveContainer" containerID="c6b03eb28aa953e5e248d21a351d6cabb5f0ab04a3eadfed9d7aa6a7403f1dd0"
Sep 17 21:18:37.428005 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:18:37.427989    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6b03eb28aa953e5e248d21a351d6cabb5f0ab04a3eadfed9d7aa6a7403f1dd0\": container with ID starting with c6b03eb28aa953e5e248d21a351d6cabb5f0ab04a3eadfed9d7aa6a7403f1dd0 not found: ID does not exist" containerID="c6b03eb28aa953e5e248d21a351d6cabb5f0ab04a3eadfed9d7aa6a7403f1dd0"
Sep 17 21:18:37.428047 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.428013    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6b03eb28aa953e5e248d21a351d6cabb5f0ab04a3eadfed9d7aa6a7403f1dd0"} err="failed to get container status \"c6b03eb28aa953e5e248d21a351d6cabb5f0ab04a3eadfed9d7aa6a7403f1dd0\": rpc error: code = NotFound desc = could not find container \"c6b03eb28aa953e5e248d21a351d6cabb5f0ab04a3eadfed9d7aa6a7403f1dd0\": container with ID starting with c6b03eb28aa953e5e248d21a351d6cabb5f0ab04a3eadfed9d7aa6a7403f1dd0 not found: ID does not exist"
Sep 17 21:18:37.428047 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.428028    2567 scope.go:117] "RemoveContainer" containerID="1a6145528b4bf7740a7efa61ace66e4886c65fe13b6240afcd869993d394b9cc"
Sep 17 21:18:37.428265 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:18:37.428247    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a6145528b4bf7740a7efa61ace66e4886c65fe13b6240afcd869993d394b9cc\": container with ID starting with 1a6145528b4bf7740a7efa61ace66e4886c65fe13b6240afcd869993d394b9cc not found: ID does not exist" containerID="1a6145528b4bf7740a7efa61ace66e4886c65fe13b6240afcd869993d394b9cc"
Sep 17 21:18:37.428335 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.428274    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a6145528b4bf7740a7efa61ace66e4886c65fe13b6240afcd869993d394b9cc"} err="failed to get container status \"1a6145528b4bf7740a7efa61ace66e4886c65fe13b6240afcd869993d394b9cc\": rpc error: code = NotFound desc = could not find container \"1a6145528b4bf7740a7efa61ace66e4886c65fe13b6240afcd869993d394b9cc\": container with ID starting with 1a6145528b4bf7740a7efa61ace66e4886c65fe13b6240afcd869993d394b9cc not found: ID does not exist"
Sep 17 21:18:37.428335 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.428299    2567 scope.go:117] "RemoveContainer" containerID="c6b03eb28aa953e5e248d21a351d6cabb5f0ab04a3eadfed9d7aa6a7403f1dd0"
Sep 17 21:18:37.428532 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.428512    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6b03eb28aa953e5e248d21a351d6cabb5f0ab04a3eadfed9d7aa6a7403f1dd0"} err="failed to get container status \"c6b03eb28aa953e5e248d21a351d6cabb5f0ab04a3eadfed9d7aa6a7403f1dd0\": rpc error: code = NotFound desc = could not find container \"c6b03eb28aa953e5e248d21a351d6cabb5f0ab04a3eadfed9d7aa6a7403f1dd0\": container with ID starting with c6b03eb28aa953e5e248d21a351d6cabb5f0ab04a3eadfed9d7aa6a7403f1dd0 not found: ID does not exist"
Sep 17 21:18:37.428579 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.428533    2567 scope.go:117] "RemoveContainer" containerID="1a6145528b4bf7740a7efa61ace66e4886c65fe13b6240afcd869993d394b9cc"
Sep 17 21:18:37.428741 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.428725    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a6145528b4bf7740a7efa61ace66e4886c65fe13b6240afcd869993d394b9cc"} err="failed to get container status \"1a6145528b4bf7740a7efa61ace66e4886c65fe13b6240afcd869993d394b9cc\": rpc error: code = NotFound desc = could not find container \"1a6145528b4bf7740a7efa61ace66e4886c65fe13b6240afcd869993d394b9cc\": container with ID starting with 1a6145528b4bf7740a7efa61ace66e4886c65fe13b6240afcd869993d394b9cc not found: ID does not exist"
Sep 17 21:18:37.439679 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.439610    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n"]
Sep 17 21:18:37.440932 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.440912    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-cipn-predictor-9774bf55d-8gg4n"]
Sep 17 21:18:37.769257 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.769218    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qqcrd/must-gather-jz9wr"]
Sep 17 21:18:37.769633 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.769567    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd99d6bd-8a68-4afb-8254-bf595ae781a3" containerName="kube-rbac-proxy"
Sep 17 21:18:37.769633 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.769580    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd99d6bd-8a68-4afb-8254-bf595ae781a3" containerName="kube-rbac-proxy"
Sep 17 21:18:37.769633 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.769589    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd99d6bd-8a68-4afb-8254-bf595ae781a3" containerName="kserve-container"
Sep 17 21:18:37.769633 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.769595    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd99d6bd-8a68-4afb-8254-bf595ae781a3" containerName="kserve-container"
Sep 17 21:18:37.769802 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.769653    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd99d6bd-8a68-4afb-8254-bf595ae781a3" containerName="kserve-container"
Sep 17 21:18:37.769802 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.769663    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd99d6bd-8a68-4afb-8254-bf595ae781a3" containerName="kube-rbac-proxy"
Sep 17 21:18:37.774101 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.774084    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qqcrd/must-gather-jz9wr"
Sep 17 21:18:37.776601 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.776581    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qqcrd\"/\"kube-root-ca.crt\""
Sep 17 21:18:37.776693 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.776647    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qqcrd\"/\"openshift-service-ca.crt\""
Sep 17 21:18:37.776740 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.776647    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qqcrd\"/\"default-dockercfg-tptwm\""
Sep 17 21:18:37.780646 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.780612    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qqcrd/must-gather-jz9wr"]
Sep 17 21:18:37.831519 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.831483    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s77rg\" (UniqueName: \"kubernetes.io/projected/886c6f44-5b4d-4eee-a227-8d5ce1cf910e-kube-api-access-s77rg\") pod \"must-gather-jz9wr\" (UID: \"886c6f44-5b4d-4eee-a227-8d5ce1cf910e\") " pod="openshift-must-gather-qqcrd/must-gather-jz9wr"
Sep 17 21:18:37.831702 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.831545    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/886c6f44-5b4d-4eee-a227-8d5ce1cf910e-must-gather-output\") pod \"must-gather-jz9wr\" (UID: \"886c6f44-5b4d-4eee-a227-8d5ce1cf910e\") " pod="openshift-must-gather-qqcrd/must-gather-jz9wr"
Sep 17 21:18:37.932027 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.931981    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/886c6f44-5b4d-4eee-a227-8d5ce1cf910e-must-gather-output\") pod \"must-gather-jz9wr\" (UID: \"886c6f44-5b4d-4eee-a227-8d5ce1cf910e\") " pod="openshift-must-gather-qqcrd/must-gather-jz9wr"
Sep 17 21:18:37.932214 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.932057    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s77rg\" (UniqueName: \"kubernetes.io/projected/886c6f44-5b4d-4eee-a227-8d5ce1cf910e-kube-api-access-s77rg\") pod \"must-gather-jz9wr\" (UID: \"886c6f44-5b4d-4eee-a227-8d5ce1cf910e\") " pod="openshift-must-gather-qqcrd/must-gather-jz9wr"
Sep 17 21:18:37.932333 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.932313    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/886c6f44-5b4d-4eee-a227-8d5ce1cf910e-must-gather-output\") pod \"must-gather-jz9wr\" (UID: \"886c6f44-5b4d-4eee-a227-8d5ce1cf910e\") " pod="openshift-must-gather-qqcrd/must-gather-jz9wr"
Sep 17 21:18:37.940408 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:37.940384    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s77rg\" (UniqueName: \"kubernetes.io/projected/886c6f44-5b4d-4eee-a227-8d5ce1cf910e-kube-api-access-s77rg\") pod \"must-gather-jz9wr\" (UID: \"886c6f44-5b4d-4eee-a227-8d5ce1cf910e\") " pod="openshift-must-gather-qqcrd/must-gather-jz9wr"
Sep 17 21:18:38.095328 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:38.095220    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qqcrd/must-gather-jz9wr"
Sep 17 21:18:38.170240 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:38.170213    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd99d6bd-8a68-4afb-8254-bf595ae781a3" path="/var/lib/kubelet/pods/dd99d6bd-8a68-4afb-8254-bf595ae781a3/volumes"
Sep 17 21:18:38.350575 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:38.350498    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:18:38.363411 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:38.363386    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qqcrd/must-gather-jz9wr"]
Sep 17 21:18:38.366229 ip-10-0-132-201 kubenswrapper[2567]: W0917 21:18:38.366199    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod886c6f44_5b4d_4eee_a227_8d5ce1cf910e.slice/crio-99da14d44e5c608a50aa1bdcab83a053f8cb2f48c7e4c4d09ce3512f076cbf73 WatchSource:0}: Error finding container 99da14d44e5c608a50aa1bdcab83a053f8cb2f48c7e4c4d09ce3512f076cbf73: Status 404 returned error can't find the container with id 99da14d44e5c608a50aa1bdcab83a053f8cb2f48c7e4c4d09ce3512f076cbf73
Sep 17 21:18:38.368396 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:38.368381    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 21:18:38.417363 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:38.417327    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qqcrd/must-gather-jz9wr" event={"ID":"886c6f44-5b4d-4eee-a227-8d5ce1cf910e","Type":"ContainerStarted","Data":"99da14d44e5c608a50aa1bdcab83a053f8cb2f48c7e4c4d09ce3512f076cbf73"}
Sep 17 21:18:45.453272 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:45.453234    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qqcrd/must-gather-jz9wr" event={"ID":"886c6f44-5b4d-4eee-a227-8d5ce1cf910e","Type":"ContainerStarted","Data":"8caeacba4663bc3240eb03918edeaddb6bc31685e4a182cf167d19c1148d7df5"}
Sep 17 21:18:45.453272 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:45.453278    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qqcrd/must-gather-jz9wr" event={"ID":"886c6f44-5b4d-4eee-a227-8d5ce1cf910e","Type":"ContainerStarted","Data":"ba4f516d6e3500277722fda357be4851fed0ecee9c2e3d72689c3ecc751803bd"}
Sep 17 21:18:45.476203 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:18:45.476151    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qqcrd/must-gather-jz9wr" podStartSLOduration=2.178144433 podStartE2EDuration="8.476136053s" totalImagesPullingTime="6.29799162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:18:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:18:45.474512213 +0000 UTC m=+1789.820114037" watchObservedRunningTime="2026-09-17 21:18:45.476136053 +0000 UTC m=+1789.821737878"
Sep 17 21:19:07.550237 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:07.550202    2567 generic.go:356] "Generic (PLEG): container finished" podID="886c6f44-5b4d-4eee-a227-8d5ce1cf910e" containerID="ba4f516d6e3500277722fda357be4851fed0ecee9c2e3d72689c3ecc751803bd" exitCode=0
Sep 17 21:19:07.550653 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:07.550276    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qqcrd/must-gather-jz9wr" event={"ID":"886c6f44-5b4d-4eee-a227-8d5ce1cf910e","Type":"ContainerDied","Data":"ba4f516d6e3500277722fda357be4851fed0ecee9c2e3d72689c3ecc751803bd"}
Sep 17 21:19:07.550703 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:07.550662    2567 scope.go:117] "RemoveContainer" containerID="ba4f516d6e3500277722fda357be4851fed0ecee9c2e3d72689c3ecc751803bd"
Sep 17 21:19:07.974287 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:07.974255    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-qqcrd_must-gather-jz9wr_886c6f44-5b4d-4eee-a227-8d5ce1cf910e/gather/0.log"
Sep 17 21:19:08.425869 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:08.425786    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:19:08.624437 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:08.624394    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:19:08.719529 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:08.719497    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:19:11.548008 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:11.547977    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-r7jbg_082f604e-de6c-43c3-bd30-02318c0e01a4/global-pull-secret-syncer/0.log"
Sep 17 21:19:11.601929 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:11.601897    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log"
Sep 17 21:19:11.704937 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:11.704895    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-201.ec2.internal_bf877a7e2526fe3fd4e830b9589fdf4b/haproxy/0.log"
Sep 17 21:19:13.399455 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:13.399423    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-qqcrd/must-gather-jz9wr"]
Sep 17 21:19:13.399876 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:13.399619    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-qqcrd/must-gather-jz9wr" podUID="886c6f44-5b4d-4eee-a227-8d5ce1cf910e" containerName="copy" containerID="cri-o://8caeacba4663bc3240eb03918edeaddb6bc31685e4a182cf167d19c1148d7df5" gracePeriod=2
Sep 17 21:19:13.404570 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:13.404541    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-qqcrd/must-gather-jz9wr"]
Sep 17 21:19:13.575992 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:13.575968    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-qqcrd_must-gather-jz9wr_886c6f44-5b4d-4eee-a227-8d5ce1cf910e/copy/0.log"
Sep 17 21:19:13.576357 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:13.576336    2567 generic.go:356] "Generic (PLEG): container finished" podID="886c6f44-5b4d-4eee-a227-8d5ce1cf910e" containerID="8caeacba4663bc3240eb03918edeaddb6bc31685e4a182cf167d19c1148d7df5" exitCode=143
Sep 17 21:19:13.627189 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:13.627168    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-qqcrd_must-gather-jz9wr_886c6f44-5b4d-4eee-a227-8d5ce1cf910e/copy/0.log"
Sep 17 21:19:13.629998 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:13.629982    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qqcrd/must-gather-jz9wr"
Sep 17 21:19:13.632480 ip-10-0-132-201 kubenswrapper[2567]: E0917 21:19:13.632455    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-jz9wr\" is forbidden: User \"system:node:ip-10-0-132-201.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-qqcrd\": no relationship found between node 'ip-10-0-132-201.ec2.internal' and this object" podUID="886c6f44-5b4d-4eee-a227-8d5ce1cf910e" pod="openshift-must-gather-qqcrd/must-gather-jz9wr"
Sep 17 21:19:13.742954 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:13.742927    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s77rg\" (UniqueName: \"kubernetes.io/projected/886c6f44-5b4d-4eee-a227-8d5ce1cf910e-kube-api-access-s77rg\") pod \"886c6f44-5b4d-4eee-a227-8d5ce1cf910e\" (UID: \"886c6f44-5b4d-4eee-a227-8d5ce1cf910e\") "
Sep 17 21:19:13.743116 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:13.743019    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/886c6f44-5b4d-4eee-a227-8d5ce1cf910e-must-gather-output\") pod \"886c6f44-5b4d-4eee-a227-8d5ce1cf910e\" (UID: \"886c6f44-5b4d-4eee-a227-8d5ce1cf910e\") "
Sep 17 21:19:13.744578 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:13.744551    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/886c6f44-5b4d-4eee-a227-8d5ce1cf910e-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "886c6f44-5b4d-4eee-a227-8d5ce1cf910e" (UID: "886c6f44-5b4d-4eee-a227-8d5ce1cf910e"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 21:19:13.745079 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:13.745057    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/886c6f44-5b4d-4eee-a227-8d5ce1cf910e-kube-api-access-s77rg" (OuterVolumeSpecName: "kube-api-access-s77rg") pod "886c6f44-5b4d-4eee-a227-8d5ce1cf910e" (UID: "886c6f44-5b4d-4eee-a227-8d5ce1cf910e"). InnerVolumeSpecName "kube-api-access-s77rg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:19:13.844034 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:13.844004    2567 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/886c6f44-5b4d-4eee-a227-8d5ce1cf910e-must-gather-output\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:19:13.844034 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:13.844031    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s77rg\" (UniqueName: \"kubernetes.io/projected/886c6f44-5b4d-4eee-a227-8d5ce1cf910e-kube-api-access-s77rg\") on node \"ip-10-0-132-201.ec2.internal\" DevicePath \"\""
Sep 17 21:19:14.171320 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:14.171240    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="886c6f44-5b4d-4eee-a227-8d5ce1cf910e" path="/var/lib/kubelet/pods/886c6f44-5b4d-4eee-a227-8d5ce1cf910e/volumes"
Sep 17 21:19:14.376518 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:14.376484    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:19:14.475529 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:14.475487    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j6w6_937db66c-5b56-482e-8402-4fd3fe1d388c/konnectivity-agent/0.log" limit=1024
Sep 17 21:19:14.581374 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:14.581349    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-qqcrd_must-gather-jz9wr_886c6f44-5b4d-4eee-a227-8d5ce1cf910e/copy/0.log"
Sep 17 21:19:14.581787 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:14.581773    2567 scope.go:117] "RemoveContainer" containerID="8caeacba4663bc3240eb03918edeaddb6bc31685e4a182cf167d19c1148d7df5"
Sep 17 21:19:14.581870 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:14.581771    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qqcrd/must-gather-jz9wr"
Sep 17 21:19:14.589400 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:14.589367    2567 scope.go:117] "RemoveContainer" containerID="ba4f516d6e3500277722fda357be4851fed0ecee9c2e3d72689c3ecc751803bd"
Sep 17 21:19:15.478513 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:15.478483    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5984459ff8-prgc8_fbf8b156-01d4-45e7-bc62-a37b96168dd0/monitoring-plugin/0.log"
Sep 17 21:19:15.726831 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:15.726803    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ddkmj_2272fdbe-f841-4620-8d45-531c5382ce08/node-exporter/0.log"
Sep 17 21:19:15.798031 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:15.798001    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ddkmj_2272fdbe-f841-4620-8d45-531c5382ce08/kube-rbac-proxy/0.log"
Sep 17 21:19:15.859656 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:15.859635    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ddkmj_2272fdbe-f841-4620-8d45-531c5382ce08/init-textfile/0.log"
Sep 17 21:19:16.110436 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:16.110356    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-c2mkj_c393e135-8471-4c74-bdd2-6f4fe2da9e26/kube-rbac-proxy-main/0.log"
Sep 17 21:19:16.144960 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:16.144932    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-c2mkj_c393e135-8471-4c74-bdd2-6f4fe2da9e26/kube-rbac-proxy-self/0.log"
Sep 17 21:19:16.172989 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:16.172962    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-c2mkj_c393e135-8471-4c74-bdd2-6f4fe2da9e26/openshift-state-metrics/0.log"
Sep 17 21:19:16.219548 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:16.219523    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_daccfc05-4529-4276-b683-78d5982cd20f/prometheus/0.log"
Sep 17 21:19:16.258168 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:16.258142    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_daccfc05-4529-4276-b683-78d5982cd20f/config-reloader/0.log"
Sep 17 21:19:16.287482 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:16.287459    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_daccfc05-4529-4276-b683-78d5982cd20f/thanos-sidecar/0.log"
Sep 17 21:19:16.322834 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:16.322808    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_daccfc05-4529-4276-b683-78d5982cd20f/kube-rbac-proxy-web/0.log"
Sep 17 21:19:16.357023 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:16.357002    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_daccfc05-4529-4276-b683-78d5982cd20f/kube-rbac-proxy/0.log"
Sep 17 21:19:16.393631 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:16.393563    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_daccfc05-4529-4276-b683-78d5982cd20f/kube-rbac-proxy-thanos/0.log"
Sep 17 21:19:16.428168 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:16.428140    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_daccfc05-4529-4276-b683-78d5982cd20f/init-config-reloader/0.log"
Sep 17 21:19:16.473889 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:16.473841    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-fvqtk_5f0637b4-c50b-4441-9dbf-774519510c83/prometheus-operator/0.log"
Sep 17 21:19:16.497776 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:16.497732    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-fvqtk_5f0637b4-c50b-4441-9dbf-774519510c83/kube-rbac-proxy/0.log"
Sep 17 21:19:16.528881 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:16.528851    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5c6747dc7b-cnr22_a5566651-8573-441b-8596-4fb8f1960f8f/prometheus-operator-admission-webhook/0.log"
Sep 17 21:19:16.653231 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:16.653142    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5dcb9bfd56-zxcf5_6f1147bc-3278-4920-882d-8a05f33bc7c7/thanos-query/0.log"
Sep 17 21:19:16.676235 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:16.676214    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5dcb9bfd56-zxcf5_6f1147bc-3278-4920-882d-8a05f33bc7c7/kube-rbac-proxy-web/0.log"
Sep 17 21:19:16.701136 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:16.701110    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5dcb9bfd56-zxcf5_6f1147bc-3278-4920-882d-8a05f33bc7c7/kube-rbac-proxy/0.log"
Sep 17 21:19:16.724274 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:16.724244    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5dcb9bfd56-zxcf5_6f1147bc-3278-4920-882d-8a05f33bc7c7/prom-label-proxy/0.log"
Sep 17 21:19:16.745056 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:16.745030    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5dcb9bfd56-zxcf5_6f1147bc-3278-4920-882d-8a05f33bc7c7/kube-rbac-proxy-rules/0.log"
Sep 17 21:19:16.768586 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:16.768557    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5dcb9bfd56-zxcf5_6f1147bc-3278-4920-882d-8a05f33bc7c7/kube-rbac-proxy-metrics/0.log"
Sep 17 21:19:18.284066 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:18.284029    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6446c8d75d-qf2vp_451c40ce-6a3d-44f0-b7e6-91fcdcb68c24/networking-console-plugin/0.log"
Sep 17 21:19:19.149670 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:19.149636    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-864d48b4c4-xcq5p_e5b8b8df-1654-4edc-87f5-db9306ddfc52/console/0.log"
Sep 17 21:19:19.533304 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:19.533269    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-zzvpz/perf-node-gather-daemonset-c7mkr"]
Sep 17 21:19:19.533699 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:19.533572    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="886c6f44-5b4d-4eee-a227-8d5ce1cf910e" containerName="gather"
Sep 17 21:19:19.533699 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:19.533583    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="886c6f44-5b4d-4eee-a227-8d5ce1cf910e" containerName="gather"
Sep 17 21:19:19.533699 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:19.533599    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="886c6f44-5b4d-4eee-a227-8d5ce1cf910e" containerName="copy"
Sep 17 21:19:19.533699 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:19.533607    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="886c6f44-5b4d-4eee-a227-8d5ce1cf910e" containerName="copy"
Sep 17 21:19:19.533699 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:19.533661    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="886c6f44-5b4d-4eee-a227-8d5ce1cf910e" containerName="copy"
Sep 17 21:19:19.533699 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:19.533673    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="886c6f44-5b4d-4eee-a227-8d5ce1cf910e" containerName="gather"
Sep 17 21:19:19.538864 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:19.538847    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zzvpz/perf-node-gather-daemonset-c7mkr"
Sep 17 21:19:19.542021 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:19.541999    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-zzvpz\"/\"default-dockercfg-g2pnc\""
Sep 17 21:19:19.542485 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:19.542472    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zzvpz\"/\"openshift-service-ca.crt\""
Sep 17 21:19:19.543160 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:19.543146    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zzvpz\"/\"kube-root-ca.crt\""
Sep 17 21:19:19.547486 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:19.547465    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zzvpz/perf-node-gather-daemonset-c7mkr"]
Sep 17 21:19:19.587218 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:19.584701    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b8bb780b-aedb-48b6-9674-365e6bb316ff-lib-modules\") pod \"perf-node-gather-daemonset-c7mkr\" (UID: \"b8bb780b-aedb-48b6-9674-365e6bb316ff\") " pod="openshift-must-gather-zzvpz/perf-node-gather-daemonset-c7mkr"
Sep 17 21:19:19.587218 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:19.584750    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b8bb780b-aedb-48b6-9674-365e6bb316ff-podres\") pod \"perf-node-gather-daemonset-c7mkr\" (UID: \"b8bb780b-aedb-48b6-9674-365e6bb316ff\") " pod="openshift-must-gather-zzvpz/perf-node-gather-daemonset-c7mkr"
Sep 17 21:19:19.587218 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:19.584794    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clwnx\" (UniqueName: \"kubernetes.io/projected/b8bb780b-aedb-48b6-9674-365e6bb316ff-kube-api-access-clwnx\") pod \"perf-node-gather-daemonset-c7mkr\" (UID: \"b8bb780b-aedb-48b6-9674-365e6bb316ff\") " pod="openshift-must-gather-zzvpz/perf-node-gather-daemonset-c7mkr"
Sep 17 21:19:19.587218 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:19.584829    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b8bb780b-aedb-48b6-9674-365e6bb316ff-sys\") pod \"perf-node-gather-daemonset-c7mkr\" (UID: \"b8bb780b-aedb-48b6-9674-365e6bb316ff\") " pod="openshift-must-gather-zzvpz/perf-node-gather-daemonset-c7mkr"
Sep 17 21:19:19.587218 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:19.584865    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b8bb780b-aedb-48b6-9674-365e6bb316ff-proc\") pod \"perf-node-gather-daemonset-c7mkr\" (UID: \"b8bb780b-aedb-48b6-9674-365e6bb316ff\") " pod="openshift-must-gather-zzvpz/perf-node-gather-daemonset-c7mkr"
Sep 17 21:19:19.685286 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:19.685245    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b8bb780b-aedb-48b6-9674-365e6bb316ff-sys\") pod \"perf-node-gather-daemonset-c7mkr\" (UID: \"b8bb780b-aedb-48b6-9674-365e6bb316ff\") " pod="openshift-must-gather-zzvpz/perf-node-gather-daemonset-c7mkr"
Sep 17 21:19:19.685286 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:19.685293    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b8bb780b-aedb-48b6-9674-365e6bb316ff-proc\") pod \"perf-node-gather-daemonset-c7mkr\" (UID: \"b8bb780b-aedb-48b6-9674-365e6bb316ff\") " pod="openshift-must-gather-zzvpz/perf-node-gather-daemonset-c7mkr"
Sep 17 21:19:19.685516 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:19.685345    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b8bb780b-aedb-48b6-9674-365e6bb316ff-sys\") pod \"perf-node-gather-daemonset-c7mkr\" (UID: \"b8bb780b-aedb-48b6-9674-365e6bb316ff\") " pod="openshift-must-gather-zzvpz/perf-node-gather-daemonset-c7mkr"
Sep 17 21:19:19.685516 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:19.685418    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b8bb780b-aedb-48b6-9674-365e6bb316ff-proc\") pod \"perf-node-gather-daemonset-c7mkr\" (UID: \"b8bb780b-aedb-48b6-9674-365e6bb316ff\") " pod="openshift-must-gather-zzvpz/perf-node-gather-daemonset-c7mkr"
Sep 17 21:19:19.685516 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:19.685356    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b8bb780b-aedb-48b6-9674-365e6bb316ff-lib-modules\") pod \"perf-node-gather-daemonset-c7mkr\" (UID: \"b8bb780b-aedb-48b6-9674-365e6bb316ff\") " pod="openshift-must-gather-zzvpz/perf-node-gather-daemonset-c7mkr"
Sep 17 21:19:19.685516 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:19.685456    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b8bb780b-aedb-48b6-9674-365e6bb316ff-lib-modules\") pod \"perf-node-gather-daemonset-c7mkr\" (UID: \"b8bb780b-aedb-48b6-9674-365e6bb316ff\") " pod="openshift-must-gather-zzvpz/perf-node-gather-daemonset-c7mkr"
Sep 17 21:19:19.685516 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:19.685479    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b8bb780b-aedb-48b6-9674-365e6bb316ff-podres\") pod \"perf-node-gather-daemonset-c7mkr\" (UID: \"b8bb780b-aedb-48b6-9674-365e6bb316ff\") " pod="openshift-must-gather-zzvpz/perf-node-gather-daemonset-c7mkr"
Sep 17 21:19:19.685516 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:19.685502    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clwnx\" (UniqueName: \"kubernetes.io/projected/b8bb780b-aedb-48b6-9674-365e6bb316ff-kube-api-access-clwnx\") pod \"perf-node-gather-daemonset-c7mkr\" (UID: \"b8bb780b-aedb-48b6-9674-365e6bb316ff\") " pod="openshift-must-gather-zzvpz/perf-node-gather-daemonset-c7mkr"
Sep 17 21:19:19.685717 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:19.685626    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b8bb780b-aedb-48b6-9674-365e6bb316ff-podres\") pod \"perf-node-gather-daemonset-c7mkr\" (UID: \"b8bb780b-aedb-48b6-9674-365e6bb316ff\") " pod="openshift-must-gather-zzvpz/perf-node-gather-daemonset-c7mkr"
Sep 17 21:19:19.694356 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:19.694337    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clwnx\" (UniqueName: \"kubernetes.io/projected/b8bb780b-aedb-48b6-9674-365e6bb316ff-kube-api-access-clwnx\") pod \"perf-node-gather-daemonset-c7mkr\" (UID: \"b8bb780b-aedb-48b6-9674-365e6bb316ff\") " pod="openshift-must-gather-zzvpz/perf-node-gather-daemonset-c7mkr"
Sep 17 21:19:19.849652 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:19.849573    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zzvpz/perf-node-gather-daemonset-c7mkr"
Sep 17 21:19:20.122886 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:20.122806    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zzvpz/perf-node-gather-daemonset-c7mkr"]
Sep 17 21:19:20.125703 ip-10-0-132-201 kubenswrapper[2567]: W0917 21:19:20.125670    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb8bb780b_aedb_48b6_9674_365e6bb316ff.slice/crio-36c4eebdb78ac41696d94bc88620bb9ed96cbb1e5a35165a88b8a9487a4a0f7e WatchSource:0}: Error finding container 36c4eebdb78ac41696d94bc88620bb9ed96cbb1e5a35165a88b8a9487a4a0f7e: Status 404 returned error can't find the container with id 36c4eebdb78ac41696d94bc88620bb9ed96cbb1e5a35165a88b8a9487a4a0f7e
Sep 17 21:19:20.606866 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:20.606830    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zzvpz/perf-node-gather-daemonset-c7mkr" event={"ID":"b8bb780b-aedb-48b6-9674-365e6bb316ff","Type":"ContainerStarted","Data":"82ec6a9de9e9173c5c2868f8fd3e4fe2fa1ee75cd680814f2537759317bf1b4d"}
Sep 17 21:19:20.607241 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:20.606887    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zzvpz/perf-node-gather-daemonset-c7mkr" event={"ID":"b8bb780b-aedb-48b6-9674-365e6bb316ff","Type":"ContainerStarted","Data":"36c4eebdb78ac41696d94bc88620bb9ed96cbb1e5a35165a88b8a9487a4a0f7e"}
Sep 17 21:19:20.607241 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:20.606907    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-zzvpz/perf-node-gather-daemonset-c7mkr"
Sep 17 21:19:20.624028 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:20.623982    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-zzvpz/perf-node-gather-daemonset-c7mkr" podStartSLOduration=1.623964856 podStartE2EDuration="1.623964856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 21:19:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:19:20.623343731 +0000 UTC m=+1824.968945577" watchObservedRunningTime="2026-09-17 21:19:20.623964856 +0000 UTC m=+1824.969566686"
Sep 17 21:19:20.649135 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:20.649106    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9v7ql_60ba7521-2f26-41e6-a17c-98e48bf83c94/dns/0.log"
Sep 17 21:19:20.671300 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:20.671271    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9v7ql_60ba7521-2f26-41e6-a17c-98e48bf83c94/kube-rbac-proxy/0.log"
Sep 17 21:19:20.762276 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:20.762229    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-prdg7_39cff161-a45c-4052-a183-e206fd6360da/dns-node-resolver/0.log"
Sep 17 21:19:21.245243 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:21.245212    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6h956_6bc4ca39-a164-4ff7-9c1c-fedf495ccb5c/node-ca/0.log"
Sep 17 21:19:22.353118 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:22.353087    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b8zb8_aa9e55ff-3a8d-477b-8165-ce0c5183b15f/serve-healthcheck-canary/0.log"
Sep 17 21:19:22.895332 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:22.895288    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nc246_5de2dae8-8948-48c7-b710-aa7f85b8abdd/kube-rbac-proxy/0.log"
Sep 17 21:19:22.913456 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:22.913429    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nc246_5de2dae8-8948-48c7-b710-aa7f85b8abdd/exporter/0.log"
Sep 17 21:19:22.931417 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:22.931385    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nc246_5de2dae8-8948-48c7-b710-aa7f85b8abdd/extractor/0.log"
Sep 17 21:19:25.291385 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:25.291352    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-66f4b8957f-jgxts_a1a4cfae-1c90-4053-91a1-035f03dcdae0/manager/0.log"
Sep 17 21:19:25.339334 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:25.339304    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-7b954ffc95-wwj5c_7ace0b21-86b2-408d-8d11-9e708a1f24af/server/0.log"
Sep 17 21:19:26.621243 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:26.621214    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-zzvpz/perf-node-gather-daemonset-c7mkr"
Sep 17 21:19:31.145158 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:31.145125    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4bpjd_f0454a2d-7d6a-4e88-bfdc-5f4dd44e754b/kube-multus/0.log"
Sep 17 21:19:31.517744 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:31.517716    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zt8xk_b1dec440-9d21-409e-a87b-7de07d399656/kube-multus-additional-cni-plugins/0.log"
Sep 17 21:19:31.538223 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:31.538196    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zt8xk_b1dec440-9d21-409e-a87b-7de07d399656/egress-router-binary-copy/0.log"
Sep 17 21:19:31.560035 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:31.560007    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zt8xk_b1dec440-9d21-409e-a87b-7de07d399656/cni-plugins/0.log"
Sep 17 21:19:31.578114 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:31.578094    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zt8xk_b1dec440-9d21-409e-a87b-7de07d399656/bond-cni-plugin/0.log"
Sep 17 21:19:31.600954 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:31.600931    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zt8xk_b1dec440-9d21-409e-a87b-7de07d399656/routeoverride-cni/0.log"
Sep 17 21:19:31.620134 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:31.620100    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zt8xk_b1dec440-9d21-409e-a87b-7de07d399656/whereabouts-cni-bincopy/0.log"
Sep 17 21:19:31.649636 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:31.649610    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zt8xk_b1dec440-9d21-409e-a87b-7de07d399656/whereabouts-cni/0.log"
Sep 17 21:19:31.859570 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:31.859493    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jhhl5_b4f3e778-0cec-45fe-be1b-a8789d63c1d7/network-metrics-daemon/0.log"
Sep 17 21:19:31.875388 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:31.875360    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jhhl5_b4f3e778-0cec-45fe-be1b-a8789d63c1d7/kube-rbac-proxy/0.log"
Sep 17 21:19:33.088251 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:33.088221    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcr8f_d17553b3-5be3-4200-adec-f293aa7a4f81/ovn-controller/0.log"
Sep 17 21:19:33.104881 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:33.104854    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcr8f_d17553b3-5be3-4200-adec-f293aa7a4f81/ovn-acl-logging/0.log"
Sep 17 21:19:33.131234 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:33.131202    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcr8f_d17553b3-5be3-4200-adec-f293aa7a4f81/kube-rbac-proxy-node/0.log"
Sep 17 21:19:33.155713 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:33.155687    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcr8f_d17553b3-5be3-4200-adec-f293aa7a4f81/kube-rbac-proxy-ovn-metrics/0.log"
Sep 17 21:19:33.171055 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:33.171030    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcr8f_d17553b3-5be3-4200-adec-f293aa7a4f81/northd/0.log"
Sep 17 21:19:33.190685 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:33.190657    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcr8f_d17553b3-5be3-4200-adec-f293aa7a4f81/nbdb/0.log"
Sep 17 21:19:33.208887 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:33.208864    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcr8f_d17553b3-5be3-4200-adec-f293aa7a4f81/sbdb/0.log"
Sep 17 21:19:33.264435 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:33.264399    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gcr8f_d17553b3-5be3-4200-adec-f293aa7a4f81/ovnkube-controller/0.log"
Sep 17 21:19:34.200170 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:34.200140    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-7cbc45fddc-ffxmr_43630f5b-7add-4f60-8fcb-425d45882a63/check-endpoints/0.log"
Sep 17 21:19:34.272131 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:34.272104    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-rb4d7_c6f7a65c-e785-4e19-aed2-5c2583881c06/network-check-target-container/0.log"
Sep 17 21:19:35.216572 ip-10-0-132-201 kubenswrapper[2567]: I0917 21:19:35.216538    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-llmv2_447ec589-f25d-47fb-9713-519a04467695/iptables-alerter/0.log"