Sep 17 16:39:57.839404 ip-10-0-130-106 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 17 16:39:57.839413 ip-10-0-130-106 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 17 16:39:57.839420 ip-10-0-130-106 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 17 16:39:57.839646 ip-10-0-130-106 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 17 16:40:09.048153 ip-10-0-130-106 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 17 16:40:09.048168 ip-10-0-130-106 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot a02cb49c9acd4e5f8338f57069bf6d29 --
Sep 17 16:42:07.768216 ip-10-0-130-106 systemd[1]: Starting Kubernetes Kubelet...
Sep 17 16:42:11.150617 ip-10-0-130-106 kubenswrapper[2586]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 16:42:11.150617 ip-10-0-130-106 kubenswrapper[2586]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 17 16:42:11.150617 ip-10-0-130-106 kubenswrapper[2586]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 16:42:11.150617 ip-10-0-130-106 kubenswrapper[2586]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 17 16:42:11.150617 ip-10-0-130-106 kubenswrapper[2586]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 16:42:11.151386 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.150694    2586 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 17 16:42:11.154640 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154623    2586 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 16:42:11.154640 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154640    2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 16:42:11.154707 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154645    2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 16:42:11.154707 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154648    2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 16:42:11.154707 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154651    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 16:42:11.154707 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154654    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 16:42:11.154707 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154657    2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 16:42:11.154707 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154660    2586 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 16:42:11.154707 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154663    2586 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 16:42:11.154707 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154665    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 16:42:11.154707 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154668    2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 16:42:11.154707 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154671    2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 16:42:11.154707 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154674    2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 16:42:11.154707 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154677    2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 16:42:11.154707 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154680    2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 16:42:11.154707 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154683    2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 16:42:11.154707 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154686    2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 16:42:11.154707 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154694    2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 16:42:11.154707 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154697    2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 16:42:11.154707 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154700    2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 16:42:11.154707 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154703    2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 16:42:11.155176 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154705    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 16:42:11.155176 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154709    2586 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 16:42:11.155176 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154712    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 16:42:11.155176 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154715    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 16:42:11.155176 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154717    2586 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 16:42:11.155176 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154720    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 16:42:11.155176 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154723    2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 16:42:11.155176 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154725    2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 16:42:11.155176 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154728    2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 16:42:11.155176 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154730    2586 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 16:42:11.155176 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154733    2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 16:42:11.155176 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154736    2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 16:42:11.155176 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154739    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 16:42:11.155176 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154741    2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 16:42:11.155176 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154744    2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 16:42:11.155176 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154746    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 16:42:11.155176 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154749    2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 16:42:11.155176 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154752    2586 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 16:42:11.155176 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154754    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 16:42:11.155176 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154757    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 16:42:11.155690 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154761    2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 16:42:11.155690 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154764    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 16:42:11.155690 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154767    2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 16:42:11.155690 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154770    2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 16:42:11.155690 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154774    2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 16:42:11.155690 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154777    2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 16:42:11.155690 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154780    2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 16:42:11.155690 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154783    2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 16:42:11.155690 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154786    2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 16:42:11.155690 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154789    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 16:42:11.155690 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154791    2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 16:42:11.155690 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154794    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 16:42:11.155690 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154797    2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 16:42:11.155690 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154802    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 16:42:11.155690 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154805    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 16:42:11.155690 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154808    2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 16:42:11.155690 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154811    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 16:42:11.155690 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154813    2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 16:42:11.155690 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154816    2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 16:42:11.156177 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154818    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 16:42:11.156177 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154821    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 16:42:11.156177 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154823    2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 16:42:11.156177 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154826    2586 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 16:42:11.156177 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154829    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 16:42:11.156177 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154831    2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 16:42:11.156177 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154834    2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 16:42:11.156177 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154836    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 16:42:11.156177 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154839    2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 16:42:11.156177 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154841    2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 16:42:11.156177 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154844    2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 16:42:11.156177 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154847    2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 16:42:11.156177 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154850    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 16:42:11.156177 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154852    2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 16:42:11.156177 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154856    2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 16:42:11.156177 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154859    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 16:42:11.156177 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154861    2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 16:42:11.156177 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154864    2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 16:42:11.156177 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154867    2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 16:42:11.156177 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154883    2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 16:42:11.156716 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154886    2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 16:42:11.156716 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154888    2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 16:42:11.156716 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154891    2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 16:42:11.156716 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154893    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 16:42:11.156716 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154896    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 16:42:11.156716 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154899    2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 16:42:11.156716 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154903    2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 16:42:11.156716 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154906    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 16:42:11.156716 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154908    2586 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 16:42:11.156716 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154911    2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 16:42:11.156716 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154914    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 16:42:11.156716 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154917    2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 16:42:11.156716 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154919    2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 16:42:11.156716 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154922    2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 16:42:11.156716 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154924    2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 16:42:11.156716 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154927    2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 16:42:11.156716 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154929    2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 16:42:11.156716 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154932    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 16:42:11.156716 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.154936    2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 16:42:11.157202 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156350    2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 16:42:11.157202 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156357    2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 16:42:11.157202 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156361    2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 16:42:11.157202 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156366    2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 16:42:11.157202 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156370    2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 16:42:11.157202 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156374    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 16:42:11.157202 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156378    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 16:42:11.157202 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156381    2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 16:42:11.157202 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156383    2586 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 16:42:11.157202 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156386    2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 16:42:11.157202 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156389    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 16:42:11.157202 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156392    2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 16:42:11.157202 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156395    2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 16:42:11.157202 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156397    2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 16:42:11.157202 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156400    2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 16:42:11.157202 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156402    2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 16:42:11.157202 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156405    2586 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 16:42:11.157202 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156407    2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 16:42:11.157202 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156410    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 16:42:11.157673 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156413    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 16:42:11.157673 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156416    2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 16:42:11.157673 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156419    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 16:42:11.157673 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156421    2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 16:42:11.157673 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156424    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 16:42:11.157673 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156427    2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 16:42:11.157673 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156429    2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 16:42:11.157673 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156431    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 16:42:11.157673 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156434    2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 16:42:11.157673 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156438    2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 16:42:11.157673 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156441    2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 16:42:11.157673 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156444    2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 16:42:11.157673 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156447    2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 16:42:11.157673 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156450    2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 16:42:11.157673 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156452    2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 16:42:11.157673 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156455    2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 16:42:11.157673 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156458    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 16:42:11.157673 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156460    2586 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 16:42:11.157673 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156463    2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 16:42:11.158152 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156467    2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 16:42:11.158152 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156470    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 16:42:11.158152 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156473    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 16:42:11.158152 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156475    2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 16:42:11.158152 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156477    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 16:42:11.158152 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156480    2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 16:42:11.158152 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156482    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 16:42:11.158152 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156485    2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 16:42:11.158152 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156487    2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 16:42:11.158152 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156490    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 16:42:11.158152 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156492    2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 16:42:11.158152 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156495    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 16:42:11.158152 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156497    2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 16:42:11.158152 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156501    2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 16:42:11.158152 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156503    2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 16:42:11.158152 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156506    2586 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 16:42:11.158152 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156509    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 16:42:11.158152 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156511    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 16:42:11.158152 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156514    2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 16:42:11.158640 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156516    2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 16:42:11.158640 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156519    2586 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 16:42:11.158640 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156521    2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 16:42:11.158640 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156524    2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 16:42:11.158640 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156526    2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 16:42:11.158640 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156529    2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 16:42:11.158640 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156531    2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 16:42:11.158640 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156534    2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 16:42:11.158640 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156536    2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 16:42:11.158640 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156538    2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 16:42:11.158640 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156541    2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 16:42:11.158640 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156543    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 16:42:11.158640 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156546    2586 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 16:42:11.158640 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156548    2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 16:42:11.158640 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156552    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 16:42:11.158640 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156554    2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 16:42:11.158640 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156557    2586 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 16:42:11.158640 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156559    2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 16:42:11.158640 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156561    2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 16:42:11.158640 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156564    2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 16:42:11.159136 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156567    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 16:42:11.159136 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156570    2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 16:42:11.159136 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156573    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 16:42:11.159136 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156576    2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 16:42:11.159136 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156578    2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 16:42:11.159136 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156582    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 16:42:11.159136 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156585    2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 16:42:11.159136 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156587    2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 16:42:11.159136 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156590    2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 16:42:11.159136 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156592    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 16:42:11.159136 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156595    2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 16:42:11.159136 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156597    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 16:42:11.159136 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156599    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 16:42:11.159136 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156602    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 16:42:11.159136 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156605    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 16:42:11.159136 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156607    2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 16:42:11.159136 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156610    2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 16:42:11.159136 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156612    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 16:42:11.159136 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156614    2586 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 16:42:11.159136 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156617    2586 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 16:42:11.159620 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156620    2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 16:42:11.159620 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.156622    2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 16:42:11.159620 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157040    2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 16:42:11.159620 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157046    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 16:42:11.159620 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157049    2586 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 16:42:11.159620 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157052    2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 16:42:11.159620 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157055    2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 16:42:11.159620 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157058    2586 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 16:42:11.159620 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157061    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 16:42:11.159620 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157063    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 16:42:11.159620 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157066    2586 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 16:42:11.159620 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157068    2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 16:42:11.159620 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157071    2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 16:42:11.159620 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157077    2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 16:42:11.159620 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157081    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 16:42:11.159620 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157084    2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 16:42:11.159620 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157087    2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 16:42:11.159620 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157090    2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 16:42:11.159620 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157094    2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 16:42:11.159620 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157097    2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 16:42:11.160141 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157100    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 16:42:11.160141 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157104    2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 16:42:11.160141 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157107    2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 16:42:11.160141 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157110    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 16:42:11.160141 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157112    2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 16:42:11.160141 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157115    2586 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 16:42:11.160141 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157117    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 16:42:11.160141 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157120    2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 16:42:11.160141 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157123    2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 16:42:11.160141 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157125    2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 16:42:11.160141 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157127    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 16:42:11.160141 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157130    2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 16:42:11.160141 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157132    2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 16:42:11.160141 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157135    2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 16:42:11.160141 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157138    2586 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 16:42:11.160141 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157140    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 16:42:11.160141 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157145    2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 16:42:11.160141 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157148    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 16:42:11.160141 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157151    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 16:42:11.160141 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157154    2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 16:42:11.160619 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157156    2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 16:42:11.160619 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157159    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 16:42:11.160619 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157161    2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 16:42:11.160619 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157164    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 16:42:11.160619 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157166    2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 16:42:11.160619 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157170    2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 16:42:11.160619 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157173    2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 16:42:11.160619 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157175    2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 16:42:11.160619 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157177    2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 16:42:11.160619 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157180    2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 16:42:11.160619 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157184    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 16:42:11.160619 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157186    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 16:42:11.160619 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157189    2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 16:42:11.160619 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157191    2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 16:42:11.160619 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157195    2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 16:42:11.160619 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157198    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 16:42:11.160619 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157202    2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 16:42:11.160619 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157205    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 16:42:11.160619 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157208    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 16:42:11.161118 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157211    2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 16:42:11.161118 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157214    2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 16:42:11.161118 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157217    2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 16:42:11.161118 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157219    2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 16:42:11.161118 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157222    2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 16:42:11.161118 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157224    2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 16:42:11.161118 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157227    2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 16:42:11.161118 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157229    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 16:42:11.161118 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157232    2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 16:42:11.161118 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157235    2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 16:42:11.161118 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157237    2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 16:42:11.161118 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157240    2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 16:42:11.161118 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157242    2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 16:42:11.161118 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157245    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 16:42:11.161118 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157248    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 16:42:11.161118 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157250    2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 16:42:11.161118 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157252    2586 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 16:42:11.161118 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157255    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 16:42:11.161118 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157259    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 16:42:11.161118 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157261    2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 16:42:11.161606 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157264    2586 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 16:42:11.161606 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157266    2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 16:42:11.161606 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157269    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 16:42:11.161606 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157271    2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 16:42:11.161606 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157274    2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 16:42:11.161606 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157276    2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 16:42:11.161606 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157279    2586 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 16:42:11.161606 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157281    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 16:42:11.161606 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157284    2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 16:42:11.161606 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157286    2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 16:42:11.161606 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157289    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 16:42:11.161606 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157291    2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 16:42:11.161606 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157294    2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 16:42:11.161606 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157296    2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 16:42:11.161606 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157299    2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 16:42:11.161606 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157301    2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 16:42:11.161606 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157304    2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 16:42:11.161606 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157306    2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 16:42:11.161606 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157309    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 16:42:11.161606 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157311    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 16:42:11.162125 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157314    2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 16:42:11.162125 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.157316    2586 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 16:42:11.162125 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157392    2586 flags.go:64] FLAG: --address="0.0.0.0"
Sep 17 16:42:11.162125 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157427    2586 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 17 16:42:11.162125 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157433    2586 flags.go:64] FLAG: --anonymous-auth="true"
Sep 17 16:42:11.162125 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157438    2586 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 17 16:42:11.162125 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157443    2586 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 17 16:42:11.162125 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157446    2586 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 17 16:42:11.162125 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157450    2586 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 17 16:42:11.162125 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157455    2586 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 17 16:42:11.162125 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157458    2586 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 17 16:42:11.162125 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157461    2586 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 17 16:42:11.162125 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157465    2586 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 17 16:42:11.162125 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157468    2586 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 17 16:42:11.162125 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157471    2586 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 17 16:42:11.162125 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157474    2586 flags.go:64] FLAG: --cgroup-root=""
Sep 17 16:42:11.162125 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157477    2586 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 17 16:42:11.162125 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157480    2586 flags.go:64] FLAG: --client-ca-file=""
Sep 17 16:42:11.162125 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157482    2586 flags.go:64] FLAG: --cloud-provider="external"
Sep 17 16:42:11.162125 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157485    2586 flags.go:64] FLAG: --cluster-dns="[]"
Sep 17 16:42:11.162125 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157490    2586 flags.go:64] FLAG: --cluster-domain=""
Sep 17 16:42:11.162125 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157492    2586 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 17 16:42:11.162125 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157495    2586 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 17 16:42:11.162125 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157498    2586 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 17 16:42:11.162696 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157502    2586 flags.go:64] FLAG: --container-log-max-files="5"
Sep 17 16:42:11.162696 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157505    2586 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 17 16:42:11.162696 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157508    2586 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 17 16:42:11.162696 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157511    2586 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 17 16:42:11.162696 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157514    2586 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 17 16:42:11.162696 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157517    2586 flags.go:64] FLAG: --contention-profiling="false"
Sep 17 16:42:11.162696 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157520    2586 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 17 16:42:11.162696 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157523    2586 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 17 16:42:11.162696 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157526    2586 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 17 16:42:11.162696 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157529    2586 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 17 16:42:11.162696 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157533    2586 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 17 16:42:11.162696 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157536    2586 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 17 16:42:11.162696 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157539    2586 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 17 16:42:11.162696 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157542    2586 flags.go:64] FLAG: --enable-load-reader="false"
Sep 17 16:42:11.162696 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157545    2586 flags.go:64] FLAG: --enable-server="true"
Sep 17 16:42:11.162696 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157548    2586 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 17 16:42:11.162696 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157553    2586 flags.go:64] FLAG: --event-burst="100"
Sep 17 16:42:11.162696 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157556    2586 flags.go:64] FLAG: --event-qps="50"
Sep 17 16:42:11.162696 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157559    2586 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 17 16:42:11.162696 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157562    2586 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 17 16:42:11.162696 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157564    2586 flags.go:64] FLAG: --eviction-hard=""
Sep 17 16:42:11.162696 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157568    2586 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 17 16:42:11.162696 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157571    2586 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 17 16:42:11.162696 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157575    2586 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 17 16:42:11.162696 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157578    2586 flags.go:64] FLAG: --eviction-soft=""
Sep 17 16:42:11.163302 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157581    2586 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 17 16:42:11.163302 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157584    2586 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 17 16:42:11.163302 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157587    2586 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 17 16:42:11.163302 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157589    2586 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 17 16:42:11.163302 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157592    2586 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 17 16:42:11.163302 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157595    2586 flags.go:64] FLAG: --fail-swap-on="true"
Sep 17 16:42:11.163302 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157598    2586 flags.go:64] FLAG: --feature-gates=""
Sep 17 16:42:11.163302 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157602    2586 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 17 16:42:11.163302 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157605    2586 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 17 16:42:11.163302 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157607    2586 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 17 16:42:11.163302 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157610    2586 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 17 16:42:11.163302 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157613    2586 flags.go:64] FLAG: --healthz-port="10248"
Sep 17 16:42:11.163302 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157616    2586 flags.go:64] FLAG: --help="false"
Sep 17 16:42:11.163302 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157619    2586 flags.go:64] FLAG: --hostname-override="ip-10-0-130-106.ec2.internal"
Sep 17 16:42:11.163302 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157622    2586 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 17 16:42:11.163302 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157624    2586 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 17 16:42:11.163302 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157627    2586 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 17 16:42:11.163302 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157630    2586 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 17 16:42:11.163302 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157633    2586 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 17 16:42:11.163302 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157636    2586 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 17 16:42:11.163302 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157639    2586 flags.go:64] FLAG: --image-service-endpoint=""
Sep 17 16:42:11.163302 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157641    2586 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 17 16:42:11.163302 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157644    2586 flags.go:64] FLAG: --kube-api-burst="100"
Sep 17 16:42:11.163917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157647    2586 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 17 16:42:11.163917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157650    2586 flags.go:64] FLAG: --kube-api-qps="50"
Sep 17 16:42:11.163917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157652    2586 flags.go:64] FLAG: --kube-reserved=""
Sep 17 16:42:11.163917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157655    2586 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 17 16:42:11.163917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157658    2586 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 17 16:42:11.163917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157661    2586 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 17 16:42:11.163917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157663    2586 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 17 16:42:11.163917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157668    2586 flags.go:64] FLAG: --lock-file=""
Sep 17 16:42:11.163917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157670    2586 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 17 16:42:11.163917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157673    2586 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 17 16:42:11.163917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157676    2586 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 17 16:42:11.163917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157681    2586 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 17 16:42:11.163917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157684    2586 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 17 16:42:11.163917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157687    2586 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 17 16:42:11.163917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157690    2586 flags.go:64] FLAG: --logging-format="text"
Sep 17 16:42:11.163917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157693    2586 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 17 16:42:11.163917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157696    2586 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 17 16:42:11.163917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157699    2586 flags.go:64] FLAG: --manifest-url=""
Sep 17 16:42:11.163917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157701    2586 flags.go:64] FLAG: --manifest-url-header=""
Sep 17 16:42:11.163917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157706    2586 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 17 16:42:11.163917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157709    2586 flags.go:64] FLAG: --max-open-files="1000000"
Sep 17 16:42:11.163917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157713    2586 flags.go:64] FLAG: --max-pods="110"
Sep 17 16:42:11.163917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157716    2586 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 17 16:42:11.163917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157719    2586 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 17 16:42:11.163917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157721    2586 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 17 16:42:11.164521 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157724    2586 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 17 16:42:11.164521 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157727    2586 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 17 16:42:11.164521 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157730    2586 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 17 16:42:11.164521 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157733    2586 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 17 16:42:11.164521 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157740    2586 flags.go:64] FLAG: --node-status-max-images="50"
Sep 17 16:42:11.164521 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157743    2586 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 17 16:42:11.164521 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157746    2586 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 17 16:42:11.164521 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157749    2586 flags.go:64] FLAG: --pod-cidr=""
Sep 17 16:42:11.164521 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157752    2586 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 17 16:42:11.164521 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157757    2586 flags.go:64] FLAG: --pod-manifest-path=""
Sep 17 16:42:11.164521 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157760    2586 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 17 16:42:11.164521 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157764    2586 flags.go:64] FLAG: --pods-per-core="0"
Sep 17 16:42:11.164521 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157767    2586 flags.go:64] FLAG: --port="10250"
Sep 17 16:42:11.164521 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157770    2586 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 17 16:42:11.164521 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157774    2586 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-005c8c1e3e3c3fbcc"
Sep 17 16:42:11.164521 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157777    2586 flags.go:64] FLAG: --qos-reserved=""
Sep 17 16:42:11.164521 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157780    2586 flags.go:64] FLAG: --read-only-port="10255"
Sep 17 16:42:11.164521 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157783    2586 flags.go:64] FLAG: --register-node="true"
Sep 17 16:42:11.164521 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157786    2586 flags.go:64] FLAG: --register-with-taints=""
Sep 17 16:42:11.164521 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157789    2586 flags.go:64] FLAG: --registry-burst="10"
Sep 17 16:42:11.164521 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157792    2586 flags.go:64] FLAG: --registry-qps="5"
Sep 17 16:42:11.164521 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157795    2586 flags.go:64] FLAG: --reserved-cpus=""
Sep 17 16:42:11.164521 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157798    2586 flags.go:64] FLAG: --reserved-memory=""
Sep 17 16:42:11.164521 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157802    2586 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 17 16:42:11.164521 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157804    2586 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 17 16:42:11.165245 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157807    2586 flags.go:64] FLAG: --rotate-certificates="false"
Sep 17 16:42:11.165245 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157810    2586 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 17 16:42:11.165245 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157813    2586 flags.go:64] FLAG: --runonce="false"
Sep 17 16:42:11.165245 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157816    2586 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 17 16:42:11.165245 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157819    2586 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 17 16:42:11.165245 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157822    2586 flags.go:64] FLAG: --seccomp-default="false"
Sep 17 16:42:11.165245 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157825    2586 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 17 16:42:11.165245 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157827    2586 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 17 16:42:11.165245 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157830    2586 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 17 16:42:11.165245 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157833    2586 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 17 16:42:11.165245 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157836    2586 flags.go:64] FLAG: --storage-driver-password="root"
Sep 17 16:42:11.165245 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157839    2586 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 17 16:42:11.165245 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157842    2586 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 17 16:42:11.165245 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157845    2586 flags.go:64] FLAG: --storage-driver-user="root"
Sep 17 16:42:11.165245 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157847    2586 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 17 16:42:11.165245 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157850    2586 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 17 16:42:11.165245 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157853    2586 flags.go:64] FLAG: --system-cgroups=""
Sep 17 16:42:11.165245 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157857    2586 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 17 16:42:11.165245 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157863    2586 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 17 16:42:11.165245 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157866    2586 flags.go:64] FLAG: --tls-cert-file=""
Sep 17 16:42:11.165245 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157882    2586 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 17 16:42:11.165245 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157889    2586 flags.go:64] FLAG: --tls-min-version=""
Sep 17 16:42:11.165245 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157892    2586 flags.go:64] FLAG: --tls-private-key-file=""
Sep 17 16:42:11.165245 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157894    2586 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 17 16:42:11.165245 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157897    2586 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 17 16:42:11.165862 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157900    2586 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 17 16:42:11.165862 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157903    2586 flags.go:64] FLAG: --v="2"
Sep 17 16:42:11.165862 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157906    2586 flags.go:64] FLAG: --version="false"
Sep 17 16:42:11.165862 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157911    2586 flags.go:64] FLAG: --vmodule=""
Sep 17 16:42:11.165862 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157915    2586 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 17 16:42:11.165862 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.157918    2586 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 17 16:42:11.165862 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158032    2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 16:42:11.165862 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158036    2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 16:42:11.165862 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158038    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 16:42:11.165862 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158041    2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 16:42:11.165862 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158044    2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 16:42:11.165862 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158046    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 16:42:11.165862 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158049    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 16:42:11.165862 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158052    2586 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 16:42:11.165862 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158054    2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 16:42:11.165862 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158057    2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 16:42:11.165862 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158061    2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 16:42:11.165862 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158065    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 16:42:11.165862 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158068    2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 16:42:11.165862 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158071    2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 16:42:11.165862 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158073    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 16:42:11.166403 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158076    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 16:42:11.166403 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158079    2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 16:42:11.166403 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158083    2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 16:42:11.166403 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158088    2586 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 16:42:11.166403 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158091    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 16:42:11.166403 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158094    2586 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 16:42:11.166403 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158097    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 16:42:11.166403 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158101    2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 16:42:11.166403 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158104    2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 16:42:11.166403 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158114    2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 16:42:11.166403 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158117    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 16:42:11.166403 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158119    2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 16:42:11.166403 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158122    2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 16:42:11.166403 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158125    2586 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 16:42:11.166403 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158127    2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 16:42:11.166403 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158130    2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 16:42:11.166403 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158133    2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 16:42:11.166403 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158136    2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 16:42:11.166403 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158138    2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 16:42:11.166905 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158141    2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 16:42:11.166905 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158144    2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 16:42:11.166905 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158147    2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 16:42:11.166905 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158149    2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 16:42:11.166905 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158152    2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 16:42:11.166905 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158155    2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 16:42:11.166905 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158158    2586 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 16:42:11.166905 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158160    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 16:42:11.166905 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158163    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 16:42:11.166905 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158166    2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 16:42:11.166905 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158168    2586 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 16:42:11.166905 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158171    2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 16:42:11.166905 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158174    2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 16:42:11.166905 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158176    2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 16:42:11.166905 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158179    2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 16:42:11.166905 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158181    2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 16:42:11.166905 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158185    2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 16:42:11.166905 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158188    2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 16:42:11.166905 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158191    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 16:42:11.166905 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158207    2586 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 16:42:11.167401 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158212    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 16:42:11.167401 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158215    2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 16:42:11.167401 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158218    2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 16:42:11.167401 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158221    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 16:42:11.167401 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158224    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 16:42:11.167401 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158227    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 16:42:11.167401 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158230    2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 16:42:11.167401 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158233    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 16:42:11.167401 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158236    2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 16:42:11.167401 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158239    2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 16:42:11.167401 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158242    2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 16:42:11.167401 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158244    2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 16:42:11.167401 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158247    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 16:42:11.167401 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158249    2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 16:42:11.167401 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158252    2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 16:42:11.167401 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158254    2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 16:42:11.167401 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158257    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 16:42:11.167401 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158260    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 16:42:11.167401 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158262    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 16:42:11.167401 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158265    2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 16:42:11.167896 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158267    2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 16:42:11.167896 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158270    2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 16:42:11.167896 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158273    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 16:42:11.167896 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158275    2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 16:42:11.167896 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158278    2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 16:42:11.167896 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158280    2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 16:42:11.167896 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158283    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 16:42:11.167896 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158285    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 16:42:11.167896 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158289    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 16:42:11.167896 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158293    2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 16:42:11.167896 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158298    2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 16:42:11.167896 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158301    2586 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 16:42:11.167896 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158309    2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 16:42:11.167896 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158312    2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 16:42:11.167896 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158315    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 16:42:11.167896 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158317    2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 16:42:11.167896 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158320    2586 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 16:42:11.167896 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158322    2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 16:42:11.167896 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158325    2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 16:42:11.167896 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158328    2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 16:42:11.168389 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158330    2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 16:42:11.168389 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158334    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 16:42:11.168389 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158336    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 16:42:11.168389 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158338    2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 16:42:11.168389 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.158341    2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 16:42:11.168389 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.158884    2586 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 16:42:11.168389 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.166948    2586 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 17 16:42:11.168389 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.166966    2586 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 17 16:42:11.168389 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.166986    2586 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 17 16:42:11.168389 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.166995    2586 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 17 16:42:11.168389 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167065    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 16:42:11.168389 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167069    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 16:42:11.168389 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167073    2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 16:42:11.168389 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167076    2586 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 16:42:11.168389 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167079    2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 16:42:11.168764 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167094    2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 16:42:11.168764 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167098    2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 16:42:11.168764 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167101    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 16:42:11.168764 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167104    2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 16:42:11.168764 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167108    2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 16:42:11.168764 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167110    2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 16:42:11.168764 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167113    2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 16:42:11.168764 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167116    2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 16:42:11.168764 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167119    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 16:42:11.168764 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167121    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 16:42:11.168764 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167124    2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 16:42:11.168764 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167126    2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 16:42:11.168764 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167129    2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 16:42:11.168764 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167131    2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 16:42:11.168764 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167134    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 16:42:11.168764 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167136    2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 16:42:11.168764 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167139    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 16:42:11.168764 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167141    2586 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 16:42:11.168764 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167143    2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 16:42:11.168764 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167146    2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 16:42:11.169374 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167149    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 16:42:11.169374 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167151    2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 16:42:11.169374 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167154    2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 16:42:11.169374 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167156    2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 16:42:11.169374 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167166    2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 16:42:11.169374 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167169    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 16:42:11.169374 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167171    2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 16:42:11.169374 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167174    2586 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 16:42:11.169374 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167176    2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 16:42:11.169374 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167179    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 16:42:11.169374 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167181    2586 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 16:42:11.169374 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167184    2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 16:42:11.169374 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167186    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 16:42:11.169374 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167189    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 16:42:11.169374 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167191    2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 16:42:11.169374 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167194    2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 16:42:11.169374 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167196    2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 16:42:11.169374 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167199    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 16:42:11.169374 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167201    2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 16:42:11.169374 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167204    2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 16:42:11.170175 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167206    2586 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 16:42:11.170175 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167208    2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 16:42:11.170175 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167211    2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 16:42:11.170175 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167213    2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 16:42:11.170175 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167216    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 16:42:11.170175 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167218    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 16:42:11.170175 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167221    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 16:42:11.170175 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167223    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 16:42:11.170175 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167226    2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 16:42:11.170175 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167228    2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 16:42:11.170175 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167231    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 16:42:11.170175 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167233    2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 16:42:11.170175 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167235    2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 16:42:11.170175 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167238    2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 16:42:11.170175 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167240    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 16:42:11.170175 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167243    2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 16:42:11.170175 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167246    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 16:42:11.170175 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167254    2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 16:42:11.170175 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167257    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 16:42:11.170175 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167260    2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 16:42:11.170824 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167262    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 16:42:11.170824 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167265    2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 16:42:11.170824 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167267    2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 16:42:11.170824 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167271    2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 16:42:11.170824 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167275    2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 16:42:11.170824 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167278    2586 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 16:42:11.170824 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167281    2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 16:42:11.170824 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167284    2586 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 16:42:11.170824 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167286    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 16:42:11.170824 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167289    2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 16:42:11.170824 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167292    2586 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 16:42:11.170824 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167295    2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 16:42:11.170824 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167298    2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 16:42:11.170824 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167302    2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 16:42:11.170824 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167305    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 16:42:11.170824 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167307    2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 16:42:11.170824 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167310    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 16:42:11.170824 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167313    2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 16:42:11.170824 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167316    2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 16:42:11.171356 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167318    2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 16:42:11.171356 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167322    2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 16:42:11.171356 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167326    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 16:42:11.171356 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167329    2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 16:42:11.171356 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167331    2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 16:42:11.171356 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167334    2586 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 16:42:11.171356 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167337    2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 16:42:11.171356 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167339    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 16:42:11.171356 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167342    2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 16:42:11.171356 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167345    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 16:42:11.171356 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167347    2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 16:42:11.171356 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167356    2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 16:42:11.171356 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167358    2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 16:42:11.171356 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167361    2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 16:42:11.171356 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167364    2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 16:42:11.171738 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.167369    2586 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 16:42:11.171738 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167540    2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 16:42:11.171738 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167546    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 16:42:11.171738 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167550    2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 16:42:11.171738 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167554    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 16:42:11.171738 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167557    2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 16:42:11.171738 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167560    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 16:42:11.171738 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167562    2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 16:42:11.171738 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167565    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 16:42:11.171738 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167568    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 16:42:11.171738 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167570    2586 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 16:42:11.171738 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167573    2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 16:42:11.171738 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167576    2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 16:42:11.171738 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167578    2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 16:42:11.172100 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167580    2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 16:42:11.172100 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167583    2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 16:42:11.172100 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167585    2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 16:42:11.172100 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167588    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 16:42:11.172100 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167590    2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 16:42:11.172100 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167592    2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 16:42:11.172100 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167595    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 16:42:11.172100 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167598    2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 16:42:11.172100 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167600    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 16:42:11.172100 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167603    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 16:42:11.172100 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167606    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 16:42:11.172100 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167608    2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 16:42:11.172100 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167611    2586 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 16:42:11.172100 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167613    2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 16:42:11.172100 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167616    2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 16:42:11.172100 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167624    2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 16:42:11.172100 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167627    2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 16:42:11.172100 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167630    2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 16:42:11.172100 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167632    2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 16:42:11.172100 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167635    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 16:42:11.172581 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167637    2586 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 16:42:11.172581 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167640    2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 16:42:11.172581 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167642    2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 16:42:11.172581 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167644    2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 16:42:11.172581 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167647    2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 16:42:11.172581 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167649    2586 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 16:42:11.172581 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167652    2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 16:42:11.172581 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167654    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 16:42:11.172581 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167657    2586 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 16:42:11.172581 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167660    2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 16:42:11.172581 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167662    2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 16:42:11.172581 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167664    2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 16:42:11.172581 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167667    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 16:42:11.172581 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167671    2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 16:42:11.172581 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167674    2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 16:42:11.172581 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167677    2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 16:42:11.172581 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167680    2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 16:42:11.172581 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167682    2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 16:42:11.172581 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167685    2586 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 16:42:11.172581 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167687    2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 16:42:11.173237 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167690    2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 16:42:11.173237 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167692    2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 16:42:11.173237 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167695    2586 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 16:42:11.173237 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167697    2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 16:42:11.173237 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167700    2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 16:42:11.173237 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167702    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 16:42:11.173237 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167705    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 16:42:11.173237 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167707    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 16:42:11.173237 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167715    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 16:42:11.173237 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167718    2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 16:42:11.173237 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167720    2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 16:42:11.173237 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167722    2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 16:42:11.173237 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167725    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 16:42:11.173237 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167727    2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 16:42:11.173237 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167730    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 16:42:11.173237 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167732    2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 16:42:11.173237 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167734    2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 16:42:11.173237 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167737    2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 16:42:11.173237 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167739    2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 16:42:11.173700 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167742    2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 16:42:11.173700 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167744    2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 16:42:11.173700 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167747    2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 16:42:11.173700 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167749    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 16:42:11.173700 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167752    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 16:42:11.173700 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167754    2586 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 16:42:11.173700 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167757    2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 16:42:11.173700 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167759    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 16:42:11.173700 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167762    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 16:42:11.173700 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167764    2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 16:42:11.173700 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167766    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 16:42:11.173700 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167769    2586 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 16:42:11.173700 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167771    2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 16:42:11.173700 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167776    2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 16:42:11.173700 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167778    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 16:42:11.173700 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167781    2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 16:42:11.173700 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167783    2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 16:42:11.173700 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167786    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 16:42:11.173700 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167789    2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 16:42:11.174167 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167793    2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 16:42:11.174167 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167796    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 16:42:11.174167 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167799    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 16:42:11.174167 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167807    2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 16:42:11.174167 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167810    2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 16:42:11.174167 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167813    2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 16:42:11.174167 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167816    2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 16:42:11.174167 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:11.167819    2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 16:42:11.174167 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.167823    2586 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 16:42:11.174167 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.167984    2586 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 17 16:42:11.174167 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.171527    2586 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 17 16:42:11.174167 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.172296    2586 server.go:1019] "Starting client certificate rotation"
Sep 17 16:42:11.174167 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.172392    2586 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 16:42:11.174495 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.172425    2586 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 16:42:11.174926 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.174912    2586 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 16:42:11.176781 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.176762    2586 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 16:42:11.193297 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.193274    2586 log.go:25] "Validated CRI v1 runtime API"
Sep 17 16:42:11.195652 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.195635    2586 log.go:25] "Validated CRI v1 image API"
Sep 17 16:42:11.196508 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.196494    2586 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 17 16:42:11.200320 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.200300    2586 fs.go:135] Filesystem UUIDs: map[299a22da-d817-4440-aac4-fb2277522555:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 7e89a18f-6c37-4398-be0e-88ceb1fc4743:/dev/nvme0n1p4]
Sep 17 16:42:11.200377 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.200320    2586 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 17 16:42:11.204567 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.204447    2586 manager.go:216] Machine: {Timestamp:2026-09-17 16:42:11.203949107 +0000 UTC m=+1.905027759 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100688 MemoryCapacity:32812150784 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a3098ab50e56c8ae6fa43ffc2a63f SystemUUID:ec2a3098-ab50-e56c-8ae6-fa43ffc2a63f BootID:a02cb49c-9acd-4e5f-8338-f57069bf6d29 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406073344 Type:vfs Inodes:4005389 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:fa:a2:b8:59 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:fa:a2:b8:59 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:62:e9:8c:db:29:a6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812150784 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 17 16:42:11.204567 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.204556    2586 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 17 16:42:11.204707 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.204636    2586 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 17 16:42:11.206189 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.206162    2586 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 17 16:42:11.206353 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.206192    2586 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-106.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 17 16:42:11.207337 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.207322    2586 topology_manager.go:138] "Creating topology manager with none policy"
Sep 17 16:42:11.207337 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.207337    2586 container_manager_linux.go:309] "Creating device plugin manager"
Sep 17 16:42:11.207405 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.207345    2586 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 16:42:11.207405 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.207354    2586 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 16:42:11.207457 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.207436    2586 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 17 16:42:11.209517 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.209500    2586 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 16:42:11.211224 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.211212    2586 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 16:42:11.211352 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.211342    2586 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 17 16:42:11.211381 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.211373    2586 kubelet.go:486] "Attempting to sync node with API server"
Sep 17 16:42:11.211408 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.211381    2586 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 17 16:42:11.211408 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.211396    2586 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 17 16:42:11.211408 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.211408    2586 kubelet.go:398] "Adding apiserver pod source"
Sep 17 16:42:11.211498 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.211417    2586 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 17 16:42:11.211946 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.211930    2586 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 16:42:11.212005 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.211956    2586 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 16:42:11.220715 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.220694    2586 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 17 16:42:11.223184 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.223166    2586 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 17 16:42:11.223270 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.223206    2586 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 17 16:42:11.224345 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.224332    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 17 16:42:11.224345 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.224351    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 17 16:42:11.224546 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.224361    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 17 16:42:11.224546 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.224370    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 17 16:42:11.224546 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.224382    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 17 16:42:11.224546 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.224390    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 17 16:42:11.224546 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.224417    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 17 16:42:11.224546 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.224425    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 17 16:42:11.224546 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.224435    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 17 16:42:11.224546 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.224442    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 17 16:42:11.224546 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.224454    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 17 16:42:11.224546 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.224462    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 17 16:42:11.224546 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.224493    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 17 16:42:11.224546 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.224501    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 17 16:42:11.226847 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.226816    2586 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-106.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 17 16:42:11.226847 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:11.226826    2586 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 17 16:42:11.227199 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:11.227176    2586 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-106.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 17 16:42:11.228375 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.228359    2586 server.go:1268] "Started kubelet"
Sep 17 16:42:11.228510 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.228453    2586 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 17 16:42:11.228621 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.228536    2586 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 17 16:42:11.228621 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.228539    2586 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 17 16:42:11.229330 ip-10-0-130-106 systemd[1]: Started Kubernetes Kubelet.
Sep 17 16:42:11.229916 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.229898    2586 server.go:310] "Adding debug handlers to kubelet server"
Sep 17 16:42:11.232389 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.232368    2586 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 17 16:42:11.232521 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:11.232506    2586 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 17 16:42:11.234958 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.234939    2586 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kmlld"
Sep 17 16:42:11.237072 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.237051    2586 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 17 16:42:11.237161 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.237053    2586 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 17 16:42:11.237219 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:11.237177    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-106.ec2.internal\" not found"
Sep 17 16:42:11.237719 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.237702    2586 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 17 16:42:11.237818 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.237753    2586 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 17 16:42:11.237818 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.237771    2586 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 17 16:42:11.237954 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.237892    2586 reconstruct.go:97] "Volume reconstruction finished"
Sep 17 16:42:11.237954 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.237901    2586 reconciler.go:29] "Reconciler: start to sync state"
Sep 17 16:42:11.238042 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.238009    2586 factory.go:55] Registering systemd factory
Sep 17 16:42:11.238042 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.238023    2586 factory.go:223] Registration of the systemd container factory successfully
Sep 17 16:42:11.238248 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.238235    2586 factory.go:153] Registering CRI-O factory
Sep 17 16:42:11.238306 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.238250    2586 factory.go:223] Registration of the crio container factory successfully
Sep 17 16:42:11.238306 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.238301    2586 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 17 16:42:11.238388 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.238358    2586 factory.go:103] Registering Raw factory
Sep 17 16:42:11.238470 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.238459    2586 manager.go:1196] Started watching for new ooms in manager
Sep 17 16:42:11.238866 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.238854    2586 manager.go:349] Starting recovery of all containers
Sep 17 16:42:11.243808 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.243780    2586 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kmlld"
Sep 17 16:42:11.248209 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:11.248178    2586 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-106.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 17 16:42:11.248361 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:11.248339    2586 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 17 16:42:11.263083 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.263058    2586 manager.go:354] Recovery completed
Sep 17 16:42:11.268572 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.268557    2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 16:42:11.270568 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.270551    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-106.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 16:42:11.270669 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.270588    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-106.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 16:42:11.270669 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.270603    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-106.ec2.internal" event="NodeHasSufficientPID"
Sep 17 16:42:11.271213 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.271195    2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 16:42:11.271213 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.271192    2586 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 17 16:42:11.271359 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.271227    2586 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 17 16:42:11.271359 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.271246    2586 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 16:42:11.272101 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.272082    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-106.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 16:42:11.272165 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.272121    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-106.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 16:42:11.272165 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.272136    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-106.ec2.internal" event="NodeHasSufficientPID"
Sep 17 16:42:11.273294 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.273279    2586 policy_none.go:49] "None policy: Start"
Sep 17 16:42:11.273294 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.273298    2586 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 17 16:42:11.273417 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.273309    2586 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 17 16:42:11.275227 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.275216    2586 policy_none.go:47] "Start"
Sep 17 16:42:11.314521 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.314503    2586 manager.go:337] "Starting Device Plugin manager"
Sep 17 16:42:11.326438 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:11.314554    2586 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 17 16:42:11.326438 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.314569    2586 server.go:87] "Starting device plugin registration server"
Sep 17 16:42:11.326438 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.314861    2586 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 17 16:42:11.326438 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.314918    2586 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 17 16:42:11.326438 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.315023    2586 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 17 16:42:11.326438 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.315118    2586 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 17 16:42:11.326438 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.315126    2586 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 17 16:42:11.326438 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:11.315566    2586 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 17 16:42:11.326438 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:11.315621    2586 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-106.ec2.internal\" not found"
Sep 17 16:42:11.326438 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.316861    2586 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 17 16:42:11.326438 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.318205    2586 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 17 16:42:11.326438 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.318248    2586 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 17 16:42:11.326438 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.318276    2586 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 17 16:42:11.326438 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:11.318314    2586 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 17 16:42:11.326438 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.320261    2586 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 16:42:11.415201 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.415100    2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 16:42:11.416300 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.416284    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-106.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 16:42:11.416404 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.416314    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-106.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 16:42:11.416404 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.416325    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-106.ec2.internal" event="NodeHasSufficientPID"
Sep 17 16:42:11.416404 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.416355    2586 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-106.ec2.internal"
Sep 17 16:42:11.418370 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.418351    2586 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-106.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-106.ec2.internal"]
Sep 17 16:42:11.418423 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.418414    2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 16:42:11.419255 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.419239    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-106.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 16:42:11.419322 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.419269    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-106.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 16:42:11.419322 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.419280    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-106.ec2.internal" event="NodeHasSufficientPID"
Sep 17 16:42:11.421542 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.421530    2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 16:42:11.421633 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.421619    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-106.ec2.internal"
Sep 17 16:42:11.421674 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.421657    2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 16:42:11.422297 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.422278    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-106.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 16:42:11.422396 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.422278    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-106.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 16:42:11.422396 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.422354    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-106.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 16:42:11.422396 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.422313    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-106.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 16:42:11.422396 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.422389    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-106.ec2.internal" event="NodeHasSufficientPID"
Sep 17 16:42:11.422535 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.422372    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-106.ec2.internal" event="NodeHasSufficientPID"
Sep 17 16:42:11.422796 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.422783    2586 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-106.ec2.internal"
Sep 17 16:42:11.422834 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:11.422803    2586 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-106.ec2.internal\": node \"ip-10-0-130-106.ec2.internal\" not found"
Sep 17 16:42:11.424670 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.424651    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-106.ec2.internal"
Sep 17 16:42:11.424745 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.424680    2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 16:42:11.425476 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.425456    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-106.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 16:42:11.425580 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.425488    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-106.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 16:42:11.425580 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.425502    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-106.ec2.internal" event="NodeHasSufficientPID"
Sep 17 16:42:11.440661 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:11.440640    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-106.ec2.internal\" not found"
Sep 17 16:42:11.443452 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:11.443438    2586 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-106.ec2.internal\" not found" node="ip-10-0-130-106.ec2.internal"
Sep 17 16:42:11.447523 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:11.447508    2586 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-106.ec2.internal\" not found" node="ip-10-0-130-106.ec2.internal"
Sep 17 16:42:11.539210 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.539178    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/882b812d92cabc3200681e9598ce9d95-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-106.ec2.internal\" (UID: \"882b812d92cabc3200681e9598ce9d95\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-106.ec2.internal"
Sep 17 16:42:11.539210 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.539211    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/882b812d92cabc3200681e9598ce9d95-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-106.ec2.internal\" (UID: \"882b812d92cabc3200681e9598ce9d95\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-106.ec2.internal"
Sep 17 16:42:11.539423 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.539253    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0052a71b7d9061ab5e61bebaf109be27-config\") pod \"kube-apiserver-proxy-ip-10-0-130-106.ec2.internal\" (UID: \"0052a71b7d9061ab5e61bebaf109be27\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-106.ec2.internal"
Sep 17 16:42:11.541277 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:11.541257    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-106.ec2.internal\" not found"
Sep 17 16:42:11.639705 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.639666    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/882b812d92cabc3200681e9598ce9d95-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-106.ec2.internal\" (UID: \"882b812d92cabc3200681e9598ce9d95\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-106.ec2.internal"
Sep 17 16:42:11.639705 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.639710    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0052a71b7d9061ab5e61bebaf109be27-config\") pod \"kube-apiserver-proxy-ip-10-0-130-106.ec2.internal\" (UID: \"0052a71b7d9061ab5e61bebaf109be27\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-106.ec2.internal"
Sep 17 16:42:11.639837 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.639726    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/882b812d92cabc3200681e9598ce9d95-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-106.ec2.internal\" (UID: \"882b812d92cabc3200681e9598ce9d95\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-106.ec2.internal"
Sep 17 16:42:11.639837 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.639765    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/882b812d92cabc3200681e9598ce9d95-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-106.ec2.internal\" (UID: \"882b812d92cabc3200681e9598ce9d95\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-106.ec2.internal"
Sep 17 16:42:11.639837 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.639771    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0052a71b7d9061ab5e61bebaf109be27-config\") pod \"kube-apiserver-proxy-ip-10-0-130-106.ec2.internal\" (UID: \"0052a71b7d9061ab5e61bebaf109be27\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-106.ec2.internal"
Sep 17 16:42:11.639837 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.639766    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/882b812d92cabc3200681e9598ce9d95-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-106.ec2.internal\" (UID: \"882b812d92cabc3200681e9598ce9d95\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-106.ec2.internal"
Sep 17 16:42:11.641773 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:11.641741    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-106.ec2.internal\" not found"
Sep 17 16:42:11.742648 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:11.742554    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-106.ec2.internal\" not found"
Sep 17 16:42:11.746736 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.746716    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-106.ec2.internal"
Sep 17 16:42:11.750827 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:11.750810    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-106.ec2.internal"
Sep 17 16:42:11.843347 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:11.843292    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-106.ec2.internal\" not found"
Sep 17 16:42:11.943938 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:11.943890    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-106.ec2.internal\" not found"
Sep 17 16:42:12.044490 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:12.044383    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-106.ec2.internal\" not found"
Sep 17 16:42:12.098985 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:12.098952    2586 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 16:42:12.145317 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:12.145274    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-106.ec2.internal\" not found"
Sep 17 16:42:12.172454 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:12.172428    2586 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 17 16:42:12.172844 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:12.172611    2586 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 17 16:42:12.172844 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:12.172628    2586 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 17 16:42:12.245921 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:12.245887    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-106.ec2.internal\" not found"
Sep 17 16:42:12.261104 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:12.261078    2586 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 16:42:12.261300 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:12.261262    2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-16 16:37:11 +0000 UTC" deadline="2028-04-09 17:00:44.759801501 +0000 UTC"
Sep 17 16:42:12.261346 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:12.261302    2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13680h18m32.498503059s"
Sep 17 16:42:12.346996 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:12.346912    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-106.ec2.internal\" not found"
Sep 17 16:42:12.383546 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:12.381484    2586 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 16:42:12.438089 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:12.438056    2586 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-106.ec2.internal"
Sep 17 16:42:12.450749 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:12.450725    2586 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 16:42:12.451640 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:12.451627    2586 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-106.ec2.internal"
Sep 17 16:42:12.459905 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:12.459858    2586 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 16:42:12.564839 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:12.564807    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod882b812d92cabc3200681e9598ce9d95.slice/crio-6c54adb669d3000e24ad4ebbc071c4d21cfd9e80fc3701cb16ea7350d837d20a WatchSource:0}: Error finding container 6c54adb669d3000e24ad4ebbc071c4d21cfd9e80fc3701cb16ea7350d837d20a: Status 404 returned error can't find the container with id 6c54adb669d3000e24ad4ebbc071c4d21cfd9e80fc3701cb16ea7350d837d20a
Sep 17 16:42:12.569534 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:12.569516    2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 16:42:12.711497 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:12.711463    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0052a71b7d9061ab5e61bebaf109be27.slice/crio-e68a261a7593b9c20a18f660d805761dca0cd4cf97125316f0aaedad68f3688a WatchSource:0}: Error finding container e68a261a7593b9c20a18f660d805761dca0cd4cf97125316f0aaedad68f3688a: Status 404 returned error can't find the container with id e68a261a7593b9c20a18f660d805761dca0cd4cf97125316f0aaedad68f3688a
Sep 17 16:42:13.014465 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.014379    2586 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 16:42:13.212801 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.212764    2586 apiserver.go:52] "Watching apiserver"
Sep 17 16:42:13.221061 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.221022    2586 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 17 16:42:13.221934 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.221558    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-22fdn","kube-system/global-pull-secret-syncer-jpzxf","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9rmt","openshift-cluster-node-tuning-operator/tuned-26cx9","openshift-image-registry/node-ca-fktbq","openshift-multus/multus-additional-cni-plugins-cbv5m","openshift-multus/network-metrics-daemon-z77cp","openshift-network-diagnostics/network-check-target-nxc7t","openshift-network-operator/iptables-alerter-8flbg","kube-system/konnectivity-agent-h8gmd","kube-system/kube-apiserver-proxy-ip-10-0-130-106.ec2.internal","openshift-dns/node-resolver-vsjlx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-106.ec2.internal","openshift-multus/multus-hw7f7"]
Sep 17 16:42:13.224510 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.224488    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:13.224634 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:13.224569    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z77cp" podUID="f3d4b959-b664-439d-b7a8-f4120b31a45d"
Sep 17 16:42:13.226653 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.226632    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:13.226751 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:13.226712    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nxc7t" podUID="c3e3bba0-3038-411b-8194-5a88b7a0f443"
Sep 17 16:42:13.229618 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.228933    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9rmt"
Sep 17 16:42:13.232130 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.231233    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.232130 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.231757    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 17 16:42:13.232130 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.231824    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 17 16:42:13.232130 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.232034    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-nxbq4\""
Sep 17 16:42:13.232130 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.232071    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 17 16:42:13.233660 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.233630    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 17 16:42:13.234702 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.233806    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8lmmr\""
Sep 17 16:42:13.234702 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.233841    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fktbq"
Sep 17 16:42:13.234702 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.234571    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 17 16:42:13.236813 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.236120    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cbv5m"
Sep 17 16:42:13.236813 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.236245    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 17 16:42:13.236813 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.236430    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 17 16:42:13.236813 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.236739    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 17 16:42:13.236813 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.236779    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ch5ht\""
Sep 17 16:42:13.237865 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.237826    2586 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 17 16:42:13.238467 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.238340    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 17 16:42:13.238586 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.238567    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8flbg"
Sep 17 16:42:13.238688 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.238619    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 17 16:42:13.238846 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.238828    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 17 16:42:13.238962 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.238904    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 17 16:42:13.239084 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.239066    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xnzzt\""
Sep 17 16:42:13.240273 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.239373    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 17 16:42:13.245697 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.242858    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.245697 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.243603    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 17 16:42:13.245697 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.243893    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 17 16:42:13.245697 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.243917    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-v9nt8\""
Sep 17 16:42:13.245697 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.244081    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 17 16:42:13.245697 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.244828    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:13.245697 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:13.244932    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jpzxf" podUID="36250538-b6ed-4b8d-8788-a205a4dd53c6"
Sep 17 16:42:13.247763 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.247282    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 17 16:42:13.247763 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.247368    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 17 16:42:13.247763 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.247479    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 17 16:42:13.247763 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.247551    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 17 16:42:13.247763 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.247643    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-h8gmd"
Sep 17 16:42:13.248097 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.247712    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 17 16:42:13.248097 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.247957    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vnmqk\""
Sep 17 16:42:13.248196 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.248114    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 17 16:42:13.248281 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.248252    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzj44\" (UniqueName: \"kubernetes.io/projected/c77bcca8-72d5-49e4-89cc-c80e9492b22f-kube-api-access-hzj44\") pod \"iptables-alerter-8flbg\" (UID: \"c77bcca8-72d5-49e4-89cc-c80e9492b22f\") " pod="openshift-network-operator/iptables-alerter-8flbg"
Sep 17 16:42:13.248342 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.248284    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b37efedc-5602-4b17-af5e-01922a3a06dc-etc-selinux\") pod \"aws-ebs-csi-driver-node-h9rmt\" (UID: \"b37efedc-5602-4b17-af5e-01922a3a06dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9rmt"
Sep 17 16:42:13.248342 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.248306    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-sys\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.248441 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.248355    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-var-lib-kubelet\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.248441 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.248389    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/50ca1290-c198-4a93-9246-9707e5df2568-etc-tuned\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.248441 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.248415    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b440e323-9b5d-4e31-b6e5-cd6c68d66451-cni-binary-copy\") pod \"multus-additional-cni-plugins-cbv5m\" (UID: \"b440e323-9b5d-4e31-b6e5-cd6c68d66451\") " pod="openshift-multus/multus-additional-cni-plugins-cbv5m"
Sep 17 16:42:13.248592 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.248443    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-etc-systemd\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.248592 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.248466    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0fc32be3-c783-4e1b-83bc-45d67217b049-host\") pod \"node-ca-fktbq\" (UID: \"0fc32be3-c783-4e1b-83bc-45d67217b049\") " pod="openshift-image-registry/node-ca-fktbq"
Sep 17 16:42:13.248592 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.248501    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b440e323-9b5d-4e31-b6e5-cd6c68d66451-cnibin\") pod \"multus-additional-cni-plugins-cbv5m\" (UID: \"b440e323-9b5d-4e31-b6e5-cd6c68d66451\") " pod="openshift-multus/multus-additional-cni-plugins-cbv5m"
Sep 17 16:42:13.248592 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.248528    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b440e323-9b5d-4e31-b6e5-cd6c68d66451-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cbv5m\" (UID: \"b440e323-9b5d-4e31-b6e5-cd6c68d66451\") " pod="openshift-multus/multus-additional-cni-plugins-cbv5m"
Sep 17 16:42:13.248592 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.248555    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9t5z\" (UniqueName: \"kubernetes.io/projected/c3e3bba0-3038-411b-8194-5a88b7a0f443-kube-api-access-k9t5z\") pod \"network-check-target-nxc7t\" (UID: \"c3e3bba0-3038-411b-8194-5a88b7a0f443\") " pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:13.248592 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.248579    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b37efedc-5602-4b17-af5e-01922a3a06dc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h9rmt\" (UID: \"b37efedc-5602-4b17-af5e-01922a3a06dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9rmt"
Sep 17 16:42:13.248917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.248615    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b37efedc-5602-4b17-af5e-01922a3a06dc-registration-dir\") pod \"aws-ebs-csi-driver-node-h9rmt\" (UID: \"b37efedc-5602-4b17-af5e-01922a3a06dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9rmt"
Sep 17 16:42:13.248917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.248677    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-etc-kubernetes\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.248917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.248703    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b440e323-9b5d-4e31-b6e5-cd6c68d66451-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cbv5m\" (UID: \"b440e323-9b5d-4e31-b6e5-cd6c68d66451\") " pod="openshift-multus/multus-additional-cni-plugins-cbv5m"
Sep 17 16:42:13.248917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.248864    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flnvd\" (UniqueName: \"kubernetes.io/projected/b440e323-9b5d-4e31-b6e5-cd6c68d66451-kube-api-access-flnvd\") pod \"multus-additional-cni-plugins-cbv5m\" (UID: \"b440e323-9b5d-4e31-b6e5-cd6c68d66451\") " pod="openshift-multus/multus-additional-cni-plugins-cbv5m"
Sep 17 16:42:13.249114 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.248921    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3d4b959-b664-439d-b7a8-f4120b31a45d-metrics-certs\") pod \"network-metrics-daemon-z77cp\" (UID: \"f3d4b959-b664-439d-b7a8-f4120b31a45d\") " pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:13.249114 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.248942    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c77bcca8-72d5-49e4-89cc-c80e9492b22f-iptables-alerter-script\") pod \"iptables-alerter-8flbg\" (UID: \"c77bcca8-72d5-49e4-89cc-c80e9492b22f\") " pod="openshift-network-operator/iptables-alerter-8flbg"
Sep 17 16:42:13.249114 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.248977    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b37efedc-5602-4b17-af5e-01922a3a06dc-device-dir\") pod \"aws-ebs-csi-driver-node-h9rmt\" (UID: \"b37efedc-5602-4b17-af5e-01922a3a06dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9rmt"
Sep 17 16:42:13.249114 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.248995    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-etc-sysctl-d\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.249114 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.249010    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-run\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.249114 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.249023    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0fc32be3-c783-4e1b-83bc-45d67217b049-serviceca\") pod \"node-ca-fktbq\" (UID: \"0fc32be3-c783-4e1b-83bc-45d67217b049\") " pod="openshift-image-registry/node-ca-fktbq"
Sep 17 16:42:13.249114 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.249048    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cptrj\" (UniqueName: \"kubernetes.io/projected/f3d4b959-b664-439d-b7a8-f4120b31a45d-kube-api-access-cptrj\") pod \"network-metrics-daemon-z77cp\" (UID: \"f3d4b959-b664-439d-b7a8-f4120b31a45d\") " pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:13.249114 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.249072    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b37efedc-5602-4b17-af5e-01922a3a06dc-sys-fs\") pod \"aws-ebs-csi-driver-node-h9rmt\" (UID: \"b37efedc-5602-4b17-af5e-01922a3a06dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9rmt"
Sep 17 16:42:13.249114 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.249092    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-etc-sysctl-conf\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.249114 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.249110    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-lib-modules\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.249632 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.249126    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/50ca1290-c198-4a93-9246-9707e5df2568-tmp\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.249632 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.249146    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b440e323-9b5d-4e31-b6e5-cd6c68d66451-system-cni-dir\") pod \"multus-additional-cni-plugins-cbv5m\" (UID: \"b440e323-9b5d-4e31-b6e5-cd6c68d66451\") " pod="openshift-multus/multus-additional-cni-plugins-cbv5m"
Sep 17 16:42:13.249632 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.249170    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b37efedc-5602-4b17-af5e-01922a3a06dc-socket-dir\") pod \"aws-ebs-csi-driver-node-h9rmt\" (UID: \"b37efedc-5602-4b17-af5e-01922a3a06dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9rmt"
Sep 17 16:42:13.249632 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.249183    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-host\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.249632 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.249198    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxv2g\" (UniqueName: \"kubernetes.io/projected/0fc32be3-c783-4e1b-83bc-45d67217b049-kube-api-access-dxv2g\") pod \"node-ca-fktbq\" (UID: \"0fc32be3-c783-4e1b-83bc-45d67217b049\") " pod="openshift-image-registry/node-ca-fktbq"
Sep 17 16:42:13.249632 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.249219    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtnf6\" (UniqueName: \"kubernetes.io/projected/b37efedc-5602-4b17-af5e-01922a3a06dc-kube-api-access-vtnf6\") pod \"aws-ebs-csi-driver-node-h9rmt\" (UID: \"b37efedc-5602-4b17-af5e-01922a3a06dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9rmt"
Sep 17 16:42:13.249632 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.249233    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-etc-modprobe-d\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.249632 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.249246    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-etc-sysconfig\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.249632 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.249259    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b440e323-9b5d-4e31-b6e5-cd6c68d66451-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cbv5m\" (UID: \"b440e323-9b5d-4e31-b6e5-cd6c68d66451\") " pod="openshift-multus/multus-additional-cni-plugins-cbv5m"
Sep 17 16:42:13.249632 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.249275    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c77bcca8-72d5-49e4-89cc-c80e9492b22f-host-slash\") pod \"iptables-alerter-8flbg\" (UID: \"c77bcca8-72d5-49e4-89cc-c80e9492b22f\") " pod="openshift-network-operator/iptables-alerter-8flbg"
Sep 17 16:42:13.249632 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.249295    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7hzr\" (UniqueName: \"kubernetes.io/projected/50ca1290-c198-4a93-9246-9707e5df2568-kube-api-access-v7hzr\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.250206 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.250046    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.250649 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.250316    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 17 16:42:13.250649 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.250431    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4hdg8\""
Sep 17 16:42:13.250919 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.250763    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 17 16:42:13.251966 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.251914    2586 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 16:42:13.252834 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.252482    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vsjlx"
Sep 17 16:42:13.252834 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.252624    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 17 16:42:13.253008 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.252931    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-68bf2\""
Sep 17 16:42:13.254748 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.254730    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 17 16:42:13.255588 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.255410    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 17 16:42:13.255588 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.255534    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pdkjt\""
Sep 17 16:42:13.277887 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.277848    2586 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qf6n7"
Sep 17 16:42:13.285558 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.285530    2586 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qf6n7"
Sep 17 16:42:13.328016 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.327937    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-106.ec2.internal" event={"ID":"0052a71b7d9061ab5e61bebaf109be27","Type":"ContainerStarted","Data":"e68a261a7593b9c20a18f660d805761dca0cd4cf97125316f0aaedad68f3688a"}
Sep 17 16:42:13.329519 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.329461    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-106.ec2.internal" event={"ID":"882b812d92cabc3200681e9598ce9d95","Type":"ContainerStarted","Data":"6c54adb669d3000e24ad4ebbc071c4d21cfd9e80fc3701cb16ea7350d837d20a"}
Sep 17 16:42:13.339505 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.339432    2586 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 17 16:42:13.349645 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.349515    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-hostroot\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.349645 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.349565    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/36250538-b6ed-4b8d-8788-a205a4dd53c6-original-pull-secret\") pod \"global-pull-secret-syncer-jpzxf\" (UID: \"36250538-b6ed-4b8d-8788-a205a4dd53c6\") " pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:13.349645 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.349593    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-systemd-units\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.349645 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.349617    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-host-cni-netd\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.349997 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.349669    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b37efedc-5602-4b17-af5e-01922a3a06dc-etc-selinux\") pod \"aws-ebs-csi-driver-node-h9rmt\" (UID: \"b37efedc-5602-4b17-af5e-01922a3a06dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9rmt"
Sep 17 16:42:13.349997 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.349723    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b440e323-9b5d-4e31-b6e5-cd6c68d66451-cni-binary-copy\") pod \"multus-additional-cni-plugins-cbv5m\" (UID: \"b440e323-9b5d-4e31-b6e5-cd6c68d66451\") " pod="openshift-multus/multus-additional-cni-plugins-cbv5m"
Sep 17 16:42:13.349997 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.349754    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-cni-binary-copy\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.349997 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.349829    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b440e323-9b5d-4e31-b6e5-cd6c68d66451-cnibin\") pod \"multus-additional-cni-plugins-cbv5m\" (UID: \"b440e323-9b5d-4e31-b6e5-cd6c68d66451\") " pod="openshift-multus/multus-additional-cni-plugins-cbv5m"
Sep 17 16:42:13.349997 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.349859    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b440e323-9b5d-4e31-b6e5-cd6c68d66451-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cbv5m\" (UID: \"b440e323-9b5d-4e31-b6e5-cd6c68d66451\") " pod="openshift-multus/multus-additional-cni-plugins-cbv5m"
Sep 17 16:42:13.349997 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.349920    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-etc-kubernetes\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.349997 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.349943    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-var-lib-openvswitch\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.349997 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.349969    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-ovnkube-config\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.349997 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.349997    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b37efedc-5602-4b17-af5e-01922a3a06dc-registration-dir\") pod \"aws-ebs-csi-driver-node-h9rmt\" (UID: \"b37efedc-5602-4b17-af5e-01922a3a06dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9rmt"
Sep 17 16:42:13.350264 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350022    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b440e323-9b5d-4e31-b6e5-cd6c68d66451-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cbv5m\" (UID: \"b440e323-9b5d-4e31-b6e5-cd6c68d66451\") " pod="openshift-multus/multus-additional-cni-plugins-cbv5m"
Sep 17 16:42:13.350264 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350098    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b37efedc-5602-4b17-af5e-01922a3a06dc-registration-dir\") pod \"aws-ebs-csi-driver-node-h9rmt\" (UID: \"b37efedc-5602-4b17-af5e-01922a3a06dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9rmt"
Sep 17 16:42:13.350264 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350133    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-multus-socket-dir-parent\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.350264 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350158    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsbk6\" (UniqueName: \"kubernetes.io/projected/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-kube-api-access-jsbk6\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.350264 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350183    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c77bcca8-72d5-49e4-89cc-c80e9492b22f-iptables-alerter-script\") pod \"iptables-alerter-8flbg\" (UID: \"c77bcca8-72d5-49e4-89cc-c80e9492b22f\") " pod="openshift-network-operator/iptables-alerter-8flbg"
Sep 17 16:42:13.350264 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350209    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b37efedc-5602-4b17-af5e-01922a3a06dc-device-dir\") pod \"aws-ebs-csi-driver-node-h9rmt\" (UID: \"b37efedc-5602-4b17-af5e-01922a3a06dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9rmt"
Sep 17 16:42:13.350264 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350234    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-run\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.350264 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350259    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0fc32be3-c783-4e1b-83bc-45d67217b049-serviceca\") pod \"node-ca-fktbq\" (UID: \"0fc32be3-c783-4e1b-83bc-45d67217b049\") " pod="openshift-image-registry/node-ca-fktbq"
Sep 17 16:42:13.350483 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350286    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b37efedc-5602-4b17-af5e-01922a3a06dc-sys-fs\") pod \"aws-ebs-csi-driver-node-h9rmt\" (UID: \"b37efedc-5602-4b17-af5e-01922a3a06dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9rmt"
Sep 17 16:42:13.350483 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350310    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-etc-sysctl-conf\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.350483 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350333    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-lib-modules\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.350483 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350373    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b440e323-9b5d-4e31-b6e5-cd6c68d66451-cni-binary-copy\") pod \"multus-additional-cni-plugins-cbv5m\" (UID: \"b440e323-9b5d-4e31-b6e5-cd6c68d66451\") " pod="openshift-multus/multus-additional-cni-plugins-cbv5m"
Sep 17 16:42:13.350483 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350427    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/50ca1290-c198-4a93-9246-9707e5df2568-tmp\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.350483 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350450    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-host-run-netns\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.350483 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350468    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-multus-conf-dir\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.350698 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350490    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9758\" (UniqueName: \"kubernetes.io/projected/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-kube-api-access-v9758\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.350698 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350514    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7615eda5-5be1-4711-a3d6-e743d87c267a-hosts-file\") pod \"node-resolver-vsjlx\" (UID: \"7615eda5-5be1-4711-a3d6-e743d87c267a\") " pod="openshift-dns/node-resolver-vsjlx"
Sep 17 16:42:13.350698 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350532    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-run\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.350698 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350537    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-host\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.350698 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350561    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97rck\" (UniqueName: \"kubernetes.io/projected/7615eda5-5be1-4711-a3d6-e743d87c267a-kube-api-access-97rck\") pod \"node-resolver-vsjlx\" (UID: \"7615eda5-5be1-4711-a3d6-e743d87c267a\") " pod="openshift-dns/node-resolver-vsjlx"
Sep 17 16:42:13.350698 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350590    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-run-openvswitch\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.350698 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350614    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-run-ovn\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.350698 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350634    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-log-socket\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.350698 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350659    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-host-run-ovn-kubernetes\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.350698 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350683    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-host-cni-bin\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.351048 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350708    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtnf6\" (UniqueName: \"kubernetes.io/projected/b37efedc-5602-4b17-af5e-01922a3a06dc-kube-api-access-vtnf6\") pod \"aws-ebs-csi-driver-node-h9rmt\" (UID: \"b37efedc-5602-4b17-af5e-01922a3a06dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9rmt"
Sep 17 16:42:13.351048 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350734    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7615eda5-5be1-4711-a3d6-e743d87c267a-tmp-dir\") pod \"node-resolver-vsjlx\" (UID: \"7615eda5-5be1-4711-a3d6-e743d87c267a\") " pod="openshift-dns/node-resolver-vsjlx"
Sep 17 16:42:13.351048 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350900    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c77bcca8-72d5-49e4-89cc-c80e9492b22f-host-slash\") pod \"iptables-alerter-8flbg\" (UID: \"c77bcca8-72d5-49e4-89cc-c80e9492b22f\") " pod="openshift-network-operator/iptables-alerter-8flbg"
Sep 17 16:42:13.351048 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350931    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7hzr\" (UniqueName: \"kubernetes.io/projected/50ca1290-c198-4a93-9246-9707e5df2568-kube-api-access-v7hzr\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.351048 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350965    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-host-run-netns\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.351048 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.350992    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/641beacd-a18d-42fe-a1f6-e75256b15892-konnectivity-ca\") pod \"konnectivity-agent-h8gmd\" (UID: \"641beacd-a18d-42fe-a1f6-e75256b15892\") " pod="kube-system/konnectivity-agent-h8gmd"
Sep 17 16:42:13.351048 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351034    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzj44\" (UniqueName: \"kubernetes.io/projected/c77bcca8-72d5-49e4-89cc-c80e9492b22f-kube-api-access-hzj44\") pod \"iptables-alerter-8flbg\" (UID: \"c77bcca8-72d5-49e4-89cc-c80e9492b22f\") " pod="openshift-network-operator/iptables-alerter-8flbg"
Sep 17 16:42:13.351364 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351059    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-sys\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.351364 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351064    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c77bcca8-72d5-49e4-89cc-c80e9492b22f-iptables-alerter-script\") pod \"iptables-alerter-8flbg\" (UID: \"c77bcca8-72d5-49e4-89cc-c80e9492b22f\") " pod="openshift-network-operator/iptables-alerter-8flbg"
Sep 17 16:42:13.351364 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351087    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-var-lib-kubelet\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.351364 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351110    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/50ca1290-c198-4a93-9246-9707e5df2568-etc-tuned\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.351364 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351131    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-host-var-lib-cni-bin\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.351364 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351151    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-env-overrides\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.351364 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351179    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-lib-modules\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.351364 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351175    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-ovnkube-script-lib\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.351364 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351217    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-etc-systemd\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.351364 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351232    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b37efedc-5602-4b17-af5e-01922a3a06dc-device-dir\") pod \"aws-ebs-csi-driver-node-h9rmt\" (UID: \"b37efedc-5602-4b17-af5e-01922a3a06dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9rmt"
Sep 17 16:42:13.351364 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351242    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0fc32be3-c783-4e1b-83bc-45d67217b049-host\") pod \"node-ca-fktbq\" (UID: \"0fc32be3-c783-4e1b-83bc-45d67217b049\") " pod="openshift-image-registry/node-ca-fktbq"
Sep 17 16:42:13.351364 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351267    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-host-run-k8s-cni-cncf-io\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.351364 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351304    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-run-systemd\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.351364 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351329    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.351364 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351356    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/36250538-b6ed-4b8d-8788-a205a4dd53c6-kubelet-config\") pod \"global-pull-secret-syncer-jpzxf\" (UID: \"36250538-b6ed-4b8d-8788-a205a4dd53c6\") " pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:13.352131 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351386    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9t5z\" (UniqueName: \"kubernetes.io/projected/c3e3bba0-3038-411b-8194-5a88b7a0f443-kube-api-access-k9t5z\") pod \"network-check-target-nxc7t\" (UID: \"c3e3bba0-3038-411b-8194-5a88b7a0f443\") " pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:13.352131 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351412    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b37efedc-5602-4b17-af5e-01922a3a06dc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h9rmt\" (UID: \"b37efedc-5602-4b17-af5e-01922a3a06dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9rmt"
Sep 17 16:42:13.352131 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351438    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-etc-kubernetes\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.352131 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351465    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flnvd\" (UniqueName: \"kubernetes.io/projected/b440e323-9b5d-4e31-b6e5-cd6c68d66451-kube-api-access-flnvd\") pod \"multus-additional-cni-plugins-cbv5m\" (UID: \"b440e323-9b5d-4e31-b6e5-cd6c68d66451\") " pod="openshift-multus/multus-additional-cni-plugins-cbv5m"
Sep 17 16:42:13.352131 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351494    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3d4b959-b664-439d-b7a8-f4120b31a45d-metrics-certs\") pod \"network-metrics-daemon-z77cp\" (UID: \"f3d4b959-b664-439d-b7a8-f4120b31a45d\") " pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:13.352131 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351520    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-etc-sysctl-d\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.352131 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351603    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0fc32be3-c783-4e1b-83bc-45d67217b049-serviceca\") pod \"node-ca-fktbq\" (UID: \"0fc32be3-c783-4e1b-83bc-45d67217b049\") " pod="openshift-image-registry/node-ca-fktbq"
Sep 17 16:42:13.352131 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351615    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cptrj\" (UniqueName: \"kubernetes.io/projected/f3d4b959-b664-439d-b7a8-f4120b31a45d-kube-api-access-cptrj\") pod \"network-metrics-daemon-z77cp\" (UID: \"f3d4b959-b664-439d-b7a8-f4120b31a45d\") " pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:13.352131 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351643    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-system-cni-dir\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.352131 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351655    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b37efedc-5602-4b17-af5e-01922a3a06dc-sys-fs\") pod \"aws-ebs-csi-driver-node-h9rmt\" (UID: \"b37efedc-5602-4b17-af5e-01922a3a06dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9rmt"
Sep 17 16:42:13.352131 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351672    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-host-var-lib-cni-multus\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.352131 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351695    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-multus-daemon-config\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.352131 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351719    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-host-slash\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.352131 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351740    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-etc-openvswitch\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.352131 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351763    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-node-log\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.352131 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351785    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-etc-sysctl-conf\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.352131 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351794    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b440e323-9b5d-4e31-b6e5-cd6c68d66451-system-cni-dir\") pod \"multus-additional-cni-plugins-cbv5m\" (UID: \"b440e323-9b5d-4e31-b6e5-cd6c68d66451\") " pod="openshift-multus/multus-additional-cni-plugins-cbv5m"
Sep 17 16:42:13.352927 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351818    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-host-run-multus-certs\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.352927 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351844    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-host-kubelet\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.352927 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351892    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-ovn-node-metrics-cert\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.352927 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351919    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/641beacd-a18d-42fe-a1f6-e75256b15892-agent-certs\") pod \"konnectivity-agent-h8gmd\" (UID: \"641beacd-a18d-42fe-a1f6-e75256b15892\") " pod="kube-system/konnectivity-agent-h8gmd"
Sep 17 16:42:13.352927 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351946    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b37efedc-5602-4b17-af5e-01922a3a06dc-socket-dir\") pod \"aws-ebs-csi-driver-node-h9rmt\" (UID: \"b37efedc-5602-4b17-af5e-01922a3a06dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9rmt"
Sep 17 16:42:13.352927 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.351973    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxv2g\" (UniqueName: \"kubernetes.io/projected/0fc32be3-c783-4e1b-83bc-45d67217b049-kube-api-access-dxv2g\") pod \"node-ca-fktbq\" (UID: \"0fc32be3-c783-4e1b-83bc-45d67217b049\") " pod="openshift-image-registry/node-ca-fktbq"
Sep 17 16:42:13.352927 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.352003    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-multus-cni-dir\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.352927 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.352031    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-host-var-lib-kubelet\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.352927 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.352057    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/36250538-b6ed-4b8d-8788-a205a4dd53c6-dbus\") pod \"global-pull-secret-syncer-jpzxf\" (UID: \"36250538-b6ed-4b8d-8788-a205a4dd53c6\") " pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:13.352927 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.352053    2586 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 17 16:42:13.352927 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.352087    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-etc-modprobe-d\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.352927 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.352097    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-host\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.352927 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.352111    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-etc-sysconfig\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.352927 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.352135    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b440e323-9b5d-4e31-b6e5-cd6c68d66451-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cbv5m\" (UID: \"b440e323-9b5d-4e31-b6e5-cd6c68d66451\") " pod="openshift-multus/multus-additional-cni-plugins-cbv5m"
Sep 17 16:42:13.352927 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.352158    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-cnibin\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.352927 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:13.352210    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 16:42:13.352927 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.352588    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b440e323-9b5d-4e31-b6e5-cd6c68d66451-system-cni-dir\") pod \"multus-additional-cni-plugins-cbv5m\" (UID: \"b440e323-9b5d-4e31-b6e5-cd6c68d66451\") " pod="openshift-multus/multus-additional-cni-plugins-cbv5m"
Sep 17 16:42:13.352927 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.352609    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c77bcca8-72d5-49e4-89cc-c80e9492b22f-host-slash\") pod \"iptables-alerter-8flbg\" (UID: \"c77bcca8-72d5-49e4-89cc-c80e9492b22f\") " pod="openshift-network-operator/iptables-alerter-8flbg"
Sep 17 16:42:13.353815 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:13.352669    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d4b959-b664-439d-b7a8-f4120b31a45d-metrics-certs podName:f3d4b959-b664-439d-b7a8-f4120b31a45d nodeName:}" failed. No retries permitted until 2026-09-17 16:42:13.852650138 +0000 UTC m=+4.553728797 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f3d4b959-b664-439d-b7a8-f4120b31a45d-metrics-certs") pod "network-metrics-daemon-z77cp" (UID: "f3d4b959-b664-439d-b7a8-f4120b31a45d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 16:42:13.353815 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.352914    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-etc-sysctl-d\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.353815 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.353295    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b440e323-9b5d-4e31-b6e5-cd6c68d66451-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cbv5m\" (UID: \"b440e323-9b5d-4e31-b6e5-cd6c68d66451\") " pod="openshift-multus/multus-additional-cni-plugins-cbv5m"
Sep 17 16:42:13.353815 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.353415    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b440e323-9b5d-4e31-b6e5-cd6c68d66451-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cbv5m\" (UID: \"b440e323-9b5d-4e31-b6e5-cd6c68d66451\") " pod="openshift-multus/multus-additional-cni-plugins-cbv5m"
Sep 17 16:42:13.353815 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.353502    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b37efedc-5602-4b17-af5e-01922a3a06dc-etc-selinux\") pod \"aws-ebs-csi-driver-node-h9rmt\" (UID: \"b37efedc-5602-4b17-af5e-01922a3a06dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9rmt"
Sep 17 16:42:13.353815 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.353617    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b440e323-9b5d-4e31-b6e5-cd6c68d66451-cnibin\") pod \"multus-additional-cni-plugins-cbv5m\" (UID: \"b440e323-9b5d-4e31-b6e5-cd6c68d66451\") " pod="openshift-multus/multus-additional-cni-plugins-cbv5m"
Sep 17 16:42:13.353815 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.353669    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-etc-systemd\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.353815 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.353699    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0fc32be3-c783-4e1b-83bc-45d67217b049-host\") pod \"node-ca-fktbq\" (UID: \"0fc32be3-c783-4e1b-83bc-45d67217b049\") " pod="openshift-image-registry/node-ca-fktbq"
Sep 17 16:42:13.355301 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.353948    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b37efedc-5602-4b17-af5e-01922a3a06dc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h9rmt\" (UID: \"b37efedc-5602-4b17-af5e-01922a3a06dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9rmt"
Sep 17 16:42:13.355301 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.353993    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-etc-kubernetes\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.355301 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.354081    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-sys\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.355301 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.354138    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-etc-sysconfig\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.355301 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.354228    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-etc-modprobe-d\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.355301 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.354335    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b37efedc-5602-4b17-af5e-01922a3a06dc-socket-dir\") pod \"aws-ebs-csi-driver-node-h9rmt\" (UID: \"b37efedc-5602-4b17-af5e-01922a3a06dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9rmt"
Sep 17 16:42:13.355301 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.354602    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b440e323-9b5d-4e31-b6e5-cd6c68d66451-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cbv5m\" (UID: \"b440e323-9b5d-4e31-b6e5-cd6c68d66451\") " pod="openshift-multus/multus-additional-cni-plugins-cbv5m"
Sep 17 16:42:13.355301 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.354850    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/50ca1290-c198-4a93-9246-9707e5df2568-var-lib-kubelet\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.359039 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.358969    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/50ca1290-c198-4a93-9246-9707e5df2568-tmp\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.361112 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.361089    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/50ca1290-c198-4a93-9246-9707e5df2568-etc-tuned\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.362787 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:13.362751    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 16:42:13.362787 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:13.362783    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 16:42:13.362937 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:13.362798    2586 projected.go:196] Error preparing data for projected volume kube-api-access-k9t5z for pod openshift-network-diagnostics/network-check-target-nxc7t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 16:42:13.362937 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:13.362867    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3e3bba0-3038-411b-8194-5a88b7a0f443-kube-api-access-k9t5z podName:c3e3bba0-3038-411b-8194-5a88b7a0f443 nodeName:}" failed. No retries permitted until 2026-09-17 16:42:13.86284815 +0000 UTC m=+4.563926797 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-k9t5z" (UniqueName: "kubernetes.io/projected/c3e3bba0-3038-411b-8194-5a88b7a0f443-kube-api-access-k9t5z") pod "network-check-target-nxc7t" (UID: "c3e3bba0-3038-411b-8194-5a88b7a0f443") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 16:42:13.365734 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.365473    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxv2g\" (UniqueName: \"kubernetes.io/projected/0fc32be3-c783-4e1b-83bc-45d67217b049-kube-api-access-dxv2g\") pod \"node-ca-fktbq\" (UID: \"0fc32be3-c783-4e1b-83bc-45d67217b049\") " pod="openshift-image-registry/node-ca-fktbq"
Sep 17 16:42:13.365734 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.365494    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzj44\" (UniqueName: \"kubernetes.io/projected/c77bcca8-72d5-49e4-89cc-c80e9492b22f-kube-api-access-hzj44\") pod \"iptables-alerter-8flbg\" (UID: \"c77bcca8-72d5-49e4-89cc-c80e9492b22f\") " pod="openshift-network-operator/iptables-alerter-8flbg"
Sep 17 16:42:13.366392 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.366346    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7hzr\" (UniqueName: \"kubernetes.io/projected/50ca1290-c198-4a93-9246-9707e5df2568-kube-api-access-v7hzr\") pod \"tuned-26cx9\" (UID: \"50ca1290-c198-4a93-9246-9707e5df2568\") " pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.367154 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.367129    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flnvd\" (UniqueName: \"kubernetes.io/projected/b440e323-9b5d-4e31-b6e5-cd6c68d66451-kube-api-access-flnvd\") pod \"multus-additional-cni-plugins-cbv5m\" (UID: \"b440e323-9b5d-4e31-b6e5-cd6c68d66451\") " pod="openshift-multus/multus-additional-cni-plugins-cbv5m"
Sep 17 16:42:13.367257 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.367229    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cptrj\" (UniqueName: \"kubernetes.io/projected/f3d4b959-b664-439d-b7a8-f4120b31a45d-kube-api-access-cptrj\") pod \"network-metrics-daemon-z77cp\" (UID: \"f3d4b959-b664-439d-b7a8-f4120b31a45d\") " pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:13.367651 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.367629    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtnf6\" (UniqueName: \"kubernetes.io/projected/b37efedc-5602-4b17-af5e-01922a3a06dc-kube-api-access-vtnf6\") pod \"aws-ebs-csi-driver-node-h9rmt\" (UID: \"b37efedc-5602-4b17-af5e-01922a3a06dc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9rmt"
Sep 17 16:42:13.453185 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453144    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-host-run-netns\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.453358 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453193    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-multus-conf-dir\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.453358 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453217    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9758\" (UniqueName: \"kubernetes.io/projected/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-kube-api-access-v9758\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.453358 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453241    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7615eda5-5be1-4711-a3d6-e743d87c267a-hosts-file\") pod \"node-resolver-vsjlx\" (UID: \"7615eda5-5be1-4711-a3d6-e743d87c267a\") " pod="openshift-dns/node-resolver-vsjlx"
Sep 17 16:42:13.453358 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453267    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97rck\" (UniqueName: \"kubernetes.io/projected/7615eda5-5be1-4711-a3d6-e743d87c267a-kube-api-access-97rck\") pod \"node-resolver-vsjlx\" (UID: \"7615eda5-5be1-4711-a3d6-e743d87c267a\") " pod="openshift-dns/node-resolver-vsjlx"
Sep 17 16:42:13.453358 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453270    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-host-run-netns\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.453358 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453291    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-run-openvswitch\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.453358 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453275    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-multus-conf-dir\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.453358 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453318    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-run-ovn\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.453358 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453341    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-log-socket\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.453358 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453348    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-run-openvswitch\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.453647 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453366    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-host-run-ovn-kubernetes\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.453647 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453391    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-host-cni-bin\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.453647 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453415    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7615eda5-5be1-4711-a3d6-e743d87c267a-tmp-dir\") pod \"node-resolver-vsjlx\" (UID: \"7615eda5-5be1-4711-a3d6-e743d87c267a\") " pod="openshift-dns/node-resolver-vsjlx"
Sep 17 16:42:13.453647 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453439    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-host-run-netns\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.453647 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453463    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/641beacd-a18d-42fe-a1f6-e75256b15892-konnectivity-ca\") pod \"konnectivity-agent-h8gmd\" (UID: \"641beacd-a18d-42fe-a1f6-e75256b15892\") " pod="kube-system/konnectivity-agent-h8gmd"
Sep 17 16:42:13.453647 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453489    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-host-var-lib-cni-bin\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.453647 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453513    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-env-overrides\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.453647 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453516    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-host-run-ovn-kubernetes\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.453647 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453537    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-ovnkube-script-lib\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.453647 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453564    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-host-run-k8s-cni-cncf-io\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.453647 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453572    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7615eda5-5be1-4711-a3d6-e743d87c267a-hosts-file\") pod \"node-resolver-vsjlx\" (UID: \"7615eda5-5be1-4711-a3d6-e743d87c267a\") " pod="openshift-dns/node-resolver-vsjlx"
Sep 17 16:42:13.453647 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453576    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-log-socket\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.453647 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453587    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-run-systemd\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.453647 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453642    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-host-var-lib-cni-bin\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.453647 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453648    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-run-ovn\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.454098 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453675    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.454098 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453702    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/36250538-b6ed-4b8d-8788-a205a4dd53c6-kubelet-config\") pod \"global-pull-secret-syncer-jpzxf\" (UID: \"36250538-b6ed-4b8d-8788-a205a4dd53c6\") " pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:13.454098 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453756    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-system-cni-dir\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.454098 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453780    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-host-var-lib-cni-multus\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.454098 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453802    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-multus-daemon-config\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.454098 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453866    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-host-slash\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.454098 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453913    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-etc-openvswitch\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.454098 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453938    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-node-log\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.454098 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453958    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7615eda5-5be1-4711-a3d6-e743d87c267a-tmp-dir\") pod \"node-resolver-vsjlx\" (UID: \"7615eda5-5be1-4711-a3d6-e743d87c267a\") " pod="openshift-dns/node-resolver-vsjlx"
Sep 17 16:42:13.454098 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453958    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.454098 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.453963    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-host-run-multus-certs\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.454098 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454013    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-host-kubelet\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.454098 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454020    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/36250538-b6ed-4b8d-8788-a205a4dd53c6-kubelet-config\") pod \"global-pull-secret-syncer-jpzxf\" (UID: \"36250538-b6ed-4b8d-8788-a205a4dd53c6\") " pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:13.454098 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454055    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-host-kubelet\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.454098 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454063    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-host-slash\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.454098 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454069    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-etc-openvswitch\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.454098 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454020    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-host-run-k8s-cni-cncf-io\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.454598 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454101    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-node-log\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.454598 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454104    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-ovn-node-metrics-cert\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.454598 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454131    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/641beacd-a18d-42fe-a1f6-e75256b15892-agent-certs\") pod \"konnectivity-agent-h8gmd\" (UID: \"641beacd-a18d-42fe-a1f6-e75256b15892\") " pod="kube-system/konnectivity-agent-h8gmd"
Sep 17 16:42:13.454598 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454136    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/641beacd-a18d-42fe-a1f6-e75256b15892-konnectivity-ca\") pod \"konnectivity-agent-h8gmd\" (UID: \"641beacd-a18d-42fe-a1f6-e75256b15892\") " pod="kube-system/konnectivity-agent-h8gmd"
Sep 17 16:42:13.454598 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454143    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-host-var-lib-cni-multus\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.454598 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454165    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-multus-cni-dir\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.454598 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454187    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-system-cni-dir\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.454598 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454190    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-host-var-lib-kubelet\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.454598 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454206    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-host-run-netns\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.454598 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454223    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/36250538-b6ed-4b8d-8788-a205a4dd53c6-dbus\") pod \"global-pull-secret-syncer-jpzxf\" (UID: \"36250538-b6ed-4b8d-8788-a205a4dd53c6\") " pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:13.454598 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454247    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-run-systemd\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.454598 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454253    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-cnibin\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.454598 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454302    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-cnibin\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.454598 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454355    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-env-overrides\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.454598 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454399    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-host-run-multus-certs\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.454598 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454438    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/36250538-b6ed-4b8d-8788-a205a4dd53c6-dbus\") pod \"global-pull-secret-syncer-jpzxf\" (UID: \"36250538-b6ed-4b8d-8788-a205a4dd53c6\") " pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:13.454598 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454452    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-multus-cni-dir\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.454598 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454224    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-host-var-lib-kubelet\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.455507 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454496    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-host-cni-bin\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.455507 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454557    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-ovnkube-script-lib\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.455507 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454601    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-hostroot\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.455507 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454636    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/36250538-b6ed-4b8d-8788-a205a4dd53c6-original-pull-secret\") pod \"global-pull-secret-syncer-jpzxf\" (UID: \"36250538-b6ed-4b8d-8788-a205a4dd53c6\") " pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:13.455507 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454662    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-systemd-units\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.455507 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454684    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-host-cni-netd\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.455507 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454700    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-hostroot\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.455507 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454705    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-cni-binary-copy\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.455507 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454753    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-etc-kubernetes\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.455507 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454782    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-var-lib-openvswitch\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.455507 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454807    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-ovnkube-config\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.455507 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454832    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-multus-socket-dir-parent\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.455507 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454857    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsbk6\" (UniqueName: \"kubernetes.io/projected/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-kube-api-access-jsbk6\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.455507 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454925    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-multus-daemon-config\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.455507 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.454978    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-etc-kubernetes\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.455507 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.455010    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-systemd-units\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.455507 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:13.455074    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 16:42:13.456681 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:13.455136    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36250538-b6ed-4b8d-8788-a205a4dd53c6-original-pull-secret podName:36250538-b6ed-4b8d-8788-a205a4dd53c6 nodeName:}" failed. No retries permitted until 2026-09-17 16:42:13.955118919 +0000 UTC m=+4.656197562 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/36250538-b6ed-4b8d-8788-a205a4dd53c6-original-pull-secret") pod "global-pull-secret-syncer-jpzxf" (UID: "36250538-b6ed-4b8d-8788-a205a4dd53c6") : object "kube-system"/"original-pull-secret" not registered
Sep 17 16:42:13.456681 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.455313    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-host-cni-netd\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.456681 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.455373    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-multus-socket-dir-parent\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.456681 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.455373    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-cni-binary-copy\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.456681 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.455424    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-var-lib-openvswitch\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.456681 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.455586    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-ovnkube-config\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.458803 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.458782    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/641beacd-a18d-42fe-a1f6-e75256b15892-agent-certs\") pod \"konnectivity-agent-h8gmd\" (UID: \"641beacd-a18d-42fe-a1f6-e75256b15892\") " pod="kube-system/konnectivity-agent-h8gmd"
Sep 17 16:42:13.462474 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.462433    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9758\" (UniqueName: \"kubernetes.io/projected/b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b-kube-api-access-v9758\") pod \"multus-hw7f7\" (UID: \"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b\") " pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.462641 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.462620    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97rck\" (UniqueName: \"kubernetes.io/projected/7615eda5-5be1-4711-a3d6-e743d87c267a-kube-api-access-97rck\") pod \"node-resolver-vsjlx\" (UID: \"7615eda5-5be1-4711-a3d6-e743d87c267a\") " pod="openshift-dns/node-resolver-vsjlx"
Sep 17 16:42:13.463284 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.463263    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-ovn-node-metrics-cert\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.471733 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.471708    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsbk6\" (UniqueName: \"kubernetes.io/projected/68559fca-75ed-4ee8-bedd-57b04fbd3f4d-kube-api-access-jsbk6\") pod \"ovnkube-node-22fdn\" (UID: \"68559fca-75ed-4ee8-bedd-57b04fbd3f4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.544755 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.544671    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9rmt"
Sep 17 16:42:13.555487 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.555299    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-26cx9"
Sep 17 16:42:13.556972 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:13.556942    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb37efedc_5602_4b17_af5e_01922a3a06dc.slice/crio-c7f81bc35952b46d76b7a1671bb23378a2a6eb1f9974698b0e4470c208833c53 WatchSource:0}: Error finding container c7f81bc35952b46d76b7a1671bb23378a2a6eb1f9974698b0e4470c208833c53: Status 404 returned error can't find the container with id c7f81bc35952b46d76b7a1671bb23378a2a6eb1f9974698b0e4470c208833c53
Sep 17 16:42:13.565240 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.565211    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fktbq"
Sep 17 16:42:13.568832 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:13.568808    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50ca1290_c198_4a93_9246_9707e5df2568.slice/crio-5de642e91c6c2c408b9a6589e15f88fad2566e2f2bd5120bb6e8b56319b6ca05 WatchSource:0}: Error finding container 5de642e91c6c2c408b9a6589e15f88fad2566e2f2bd5120bb6e8b56319b6ca05: Status 404 returned error can't find the container with id 5de642e91c6c2c408b9a6589e15f88fad2566e2f2bd5120bb6e8b56319b6ca05
Sep 17 16:42:13.574635 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:13.574607    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fc32be3_c783_4e1b_83bc_45d67217b049.slice/crio-b31d49783216f4b6075b9fc6a1d8ea6791c3efeeb93a3a09f39a979f45fb4f74 WatchSource:0}: Error finding container b31d49783216f4b6075b9fc6a1d8ea6791c3efeeb93a3a09f39a979f45fb4f74: Status 404 returned error can't find the container with id b31d49783216f4b6075b9fc6a1d8ea6791c3efeeb93a3a09f39a979f45fb4f74
Sep 17 16:42:13.574749 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.574697    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cbv5m"
Sep 17 16:42:13.582242 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.582218    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8flbg"
Sep 17 16:42:13.592125 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.592101    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:13.592826 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:13.592755    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc77bcca8_72d5_49e4_89cc_c80e9492b22f.slice/crio-cd10bd820cc42121c46e7d734fe033e93cbef2b7aee9ab383b9432b6ff276f55 WatchSource:0}: Error finding container cd10bd820cc42121c46e7d734fe033e93cbef2b7aee9ab383b9432b6ff276f55: Status 404 returned error can't find the container with id cd10bd820cc42121c46e7d734fe033e93cbef2b7aee9ab383b9432b6ff276f55
Sep 17 16:42:13.600098 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.600077    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-h8gmd"
Sep 17 16:42:13.602621 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:13.602593    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68559fca_75ed_4ee8_bedd_57b04fbd3f4d.slice/crio-4ce170ca2fc6d43f6a93e9008a72ab31b40dd8cdf791954f46591f39ba4c9669 WatchSource:0}: Error finding container 4ce170ca2fc6d43f6a93e9008a72ab31b40dd8cdf791954f46591f39ba4c9669: Status 404 returned error can't find the container with id 4ce170ca2fc6d43f6a93e9008a72ab31b40dd8cdf791954f46591f39ba4c9669
Sep 17 16:42:13.609100 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.609082    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hw7f7"
Sep 17 16:42:13.610241 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:13.610212    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod641beacd_a18d_42fe_a1f6_e75256b15892.slice/crio-9201d1005a114e592be5d3f61a66a0500a19bc854dd172783682e8f1d631d442 WatchSource:0}: Error finding container 9201d1005a114e592be5d3f61a66a0500a19bc854dd172783682e8f1d631d442: Status 404 returned error can't find the container with id 9201d1005a114e592be5d3f61a66a0500a19bc854dd172783682e8f1d631d442
Sep 17 16:42:13.615289 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.615111    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vsjlx"
Sep 17 16:42:13.624057 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:13.624028    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9a0d1a1_6538_4d4f_be6a_6b8208e2ec6b.slice/crio-5852327cc313e3677c67be1192107e55db944ef66d21616efd7af746605e53c6 WatchSource:0}: Error finding container 5852327cc313e3677c67be1192107e55db944ef66d21616efd7af746605e53c6: Status 404 returned error can't find the container with id 5852327cc313e3677c67be1192107e55db944ef66d21616efd7af746605e53c6
Sep 17 16:42:13.627627 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:42:13.627598    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7615eda5_5be1_4711_a3d6_e743d87c267a.slice/crio-29c824dc17bfdf92b3b24a8540b11fe8b3c80c1b026b254b85b4b0fef69f297d WatchSource:0}: Error finding container 29c824dc17bfdf92b3b24a8540b11fe8b3c80c1b026b254b85b4b0fef69f297d: Status 404 returned error can't find the container with id 29c824dc17bfdf92b3b24a8540b11fe8b3c80c1b026b254b85b4b0fef69f297d
Sep 17 16:42:13.858004 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.857909    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3d4b959-b664-439d-b7a8-f4120b31a45d-metrics-certs\") pod \"network-metrics-daemon-z77cp\" (UID: \"f3d4b959-b664-439d-b7a8-f4120b31a45d\") " pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:13.858172 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:13.858082    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 16:42:13.858172 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:13.858155    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d4b959-b664-439d-b7a8-f4120b31a45d-metrics-certs podName:f3d4b959-b664-439d-b7a8-f4120b31a45d nodeName:}" failed. No retries permitted until 2026-09-17 16:42:14.858138586 +0000 UTC m=+5.559217230 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f3d4b959-b664-439d-b7a8-f4120b31a45d-metrics-certs") pod "network-metrics-daemon-z77cp" (UID: "f3d4b959-b664-439d-b7a8-f4120b31a45d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 16:42:13.959068 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.959031    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9t5z\" (UniqueName: \"kubernetes.io/projected/c3e3bba0-3038-411b-8194-5a88b7a0f443-kube-api-access-k9t5z\") pod \"network-check-target-nxc7t\" (UID: \"c3e3bba0-3038-411b-8194-5a88b7a0f443\") " pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:13.959215 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:13.959086    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/36250538-b6ed-4b8d-8788-a205a4dd53c6-original-pull-secret\") pod \"global-pull-secret-syncer-jpzxf\" (UID: \"36250538-b6ed-4b8d-8788-a205a4dd53c6\") " pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:13.959215 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:13.959180    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 16:42:13.959215 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:13.959203    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 16:42:13.959335 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:13.959224    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 16:42:13.959335 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:13.959234    2586 projected.go:196] Error preparing data for projected volume kube-api-access-k9t5z for pod openshift-network-diagnostics/network-check-target-nxc7t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 16:42:13.959335 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:13.959244    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36250538-b6ed-4b8d-8788-a205a4dd53c6-original-pull-secret podName:36250538-b6ed-4b8d-8788-a205a4dd53c6 nodeName:}" failed. No retries permitted until 2026-09-17 16:42:14.959224004 +0000 UTC m=+5.660302644 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/36250538-b6ed-4b8d-8788-a205a4dd53c6-original-pull-secret") pod "global-pull-secret-syncer-jpzxf" (UID: "36250538-b6ed-4b8d-8788-a205a4dd53c6") : object "kube-system"/"original-pull-secret" not registered
Sep 17 16:42:13.959335 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:13.959278    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3e3bba0-3038-411b-8194-5a88b7a0f443-kube-api-access-k9t5z podName:c3e3bba0-3038-411b-8194-5a88b7a0f443 nodeName:}" failed. No retries permitted until 2026-09-17 16:42:14.959262665 +0000 UTC m=+5.660341304 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-k9t5z" (UniqueName: "kubernetes.io/projected/c3e3bba0-3038-411b-8194-5a88b7a0f443-kube-api-access-k9t5z") pod "network-check-target-nxc7t" (UID: "c3e3bba0-3038-411b-8194-5a88b7a0f443") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 16:42:14.289667 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:14.289554    2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 16:37:13 +0000 UTC" deadline="2028-04-02 16:04:02.06406535 +0000 UTC"
Sep 17 16:42:14.289667 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:14.289615    2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13511h21m47.774454541s"
Sep 17 16:42:14.334299 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:14.334259    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cbv5m" event={"ID":"b440e323-9b5d-4e31-b6e5-cd6c68d66451","Type":"ContainerStarted","Data":"abd8216d6e6c026e8459e298939e9a30b400078d2d6867feab83a73b4acf5195"}
Sep 17 16:42:14.336532 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:14.336499    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fktbq" event={"ID":"0fc32be3-c783-4e1b-83bc-45d67217b049","Type":"ContainerStarted","Data":"b31d49783216f4b6075b9fc6a1d8ea6791c3efeeb93a3a09f39a979f45fb4f74"}
Sep 17 16:42:14.338929 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:14.338896    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vsjlx" event={"ID":"7615eda5-5be1-4711-a3d6-e743d87c267a","Type":"ContainerStarted","Data":"29c824dc17bfdf92b3b24a8540b11fe8b3c80c1b026b254b85b4b0fef69f297d"}
Sep 17 16:42:14.341370 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:14.341345    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hw7f7" event={"ID":"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b","Type":"ContainerStarted","Data":"5852327cc313e3677c67be1192107e55db944ef66d21616efd7af746605e53c6"}
Sep 17 16:42:14.343083 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:14.343020    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-h8gmd" event={"ID":"641beacd-a18d-42fe-a1f6-e75256b15892","Type":"ContainerStarted","Data":"9201d1005a114e592be5d3f61a66a0500a19bc854dd172783682e8f1d631d442"}
Sep 17 16:42:14.345119 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:14.345094    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-22fdn" event={"ID":"68559fca-75ed-4ee8-bedd-57b04fbd3f4d","Type":"ContainerStarted","Data":"4ce170ca2fc6d43f6a93e9008a72ab31b40dd8cdf791954f46591f39ba4c9669"}
Sep 17 16:42:14.347344 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:14.347289    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-26cx9" event={"ID":"50ca1290-c198-4a93-9246-9707e5df2568","Type":"ContainerStarted","Data":"5de642e91c6c2c408b9a6589e15f88fad2566e2f2bd5120bb6e8b56319b6ca05"}
Sep 17 16:42:14.349018 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:14.348949    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9rmt" event={"ID":"b37efedc-5602-4b17-af5e-01922a3a06dc","Type":"ContainerStarted","Data":"c7f81bc35952b46d76b7a1671bb23378a2a6eb1f9974698b0e4470c208833c53"}
Sep 17 16:42:14.350564 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:14.350537    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8flbg" event={"ID":"c77bcca8-72d5-49e4-89cc-c80e9492b22f","Type":"ContainerStarted","Data":"cd10bd820cc42121c46e7d734fe033e93cbef2b7aee9ab383b9432b6ff276f55"}
Sep 17 16:42:14.767354 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:14.767011    2586 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 16:42:14.867809 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:14.867771    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3d4b959-b664-439d-b7a8-f4120b31a45d-metrics-certs\") pod \"network-metrics-daemon-z77cp\" (UID: \"f3d4b959-b664-439d-b7a8-f4120b31a45d\") " pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:14.867997 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:14.867949    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 16:42:14.867997 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:14.867999    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d4b959-b664-439d-b7a8-f4120b31a45d-metrics-certs podName:f3d4b959-b664-439d-b7a8-f4120b31a45d nodeName:}" failed. No retries permitted until 2026-09-17 16:42:16.867985605 +0000 UTC m=+7.569064244 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f3d4b959-b664-439d-b7a8-f4120b31a45d-metrics-certs") pod "network-metrics-daemon-z77cp" (UID: "f3d4b959-b664-439d-b7a8-f4120b31a45d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 16:42:14.969513 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:14.968644    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9t5z\" (UniqueName: \"kubernetes.io/projected/c3e3bba0-3038-411b-8194-5a88b7a0f443-kube-api-access-k9t5z\") pod \"network-check-target-nxc7t\" (UID: \"c3e3bba0-3038-411b-8194-5a88b7a0f443\") " pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:14.969513 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:14.968720    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/36250538-b6ed-4b8d-8788-a205a4dd53c6-original-pull-secret\") pod \"global-pull-secret-syncer-jpzxf\" (UID: \"36250538-b6ed-4b8d-8788-a205a4dd53c6\") " pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:14.969513 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:14.968861    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 16:42:14.969513 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:14.968948    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36250538-b6ed-4b8d-8788-a205a4dd53c6-original-pull-secret podName:36250538-b6ed-4b8d-8788-a205a4dd53c6 nodeName:}" failed. No retries permitted until 2026-09-17 16:42:16.968928629 +0000 UTC m=+7.670007280 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/36250538-b6ed-4b8d-8788-a205a4dd53c6-original-pull-secret") pod "global-pull-secret-syncer-jpzxf" (UID: "36250538-b6ed-4b8d-8788-a205a4dd53c6") : object "kube-system"/"original-pull-secret" not registered
Sep 17 16:42:14.969513 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:14.969369    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 16:42:14.969513 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:14.969387    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 16:42:14.969513 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:14.969400    2586 projected.go:196] Error preparing data for projected volume kube-api-access-k9t5z for pod openshift-network-diagnostics/network-check-target-nxc7t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 16:42:14.969513 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:14.969475    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3e3bba0-3038-411b-8194-5a88b7a0f443-kube-api-access-k9t5z podName:c3e3bba0-3038-411b-8194-5a88b7a0f443 nodeName:}" failed. No retries permitted until 2026-09-17 16:42:16.969434257 +0000 UTC m=+7.670512900 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-k9t5z" (UniqueName: "kubernetes.io/projected/c3e3bba0-3038-411b-8194-5a88b7a0f443-kube-api-access-k9t5z") pod "network-check-target-nxc7t" (UID: "c3e3bba0-3038-411b-8194-5a88b7a0f443") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 16:42:15.290308 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:15.290271    2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 16:37:13 +0000 UTC" deadline="2028-02-21 12:04:07.206205674 +0000 UTC"
Sep 17 16:42:15.290679 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:15.290309    2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12523h21m51.915901416s"
Sep 17 16:42:15.320976 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:15.320864    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:15.321137 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:15.321024    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jpzxf" podUID="36250538-b6ed-4b8d-8788-a205a4dd53c6"
Sep 17 16:42:15.321768 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:15.321742    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:15.321906 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:15.321846    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nxc7t" podUID="c3e3bba0-3038-411b-8194-5a88b7a0f443"
Sep 17 16:42:15.321987 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:15.321932    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:15.322042 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:15.322021    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z77cp" podUID="f3d4b959-b664-439d-b7a8-f4120b31a45d"
Sep 17 16:42:16.886533 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:16.885919    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3d4b959-b664-439d-b7a8-f4120b31a45d-metrics-certs\") pod \"network-metrics-daemon-z77cp\" (UID: \"f3d4b959-b664-439d-b7a8-f4120b31a45d\") " pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:16.886533 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:16.886111    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 16:42:16.886533 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:16.886175    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d4b959-b664-439d-b7a8-f4120b31a45d-metrics-certs podName:f3d4b959-b664-439d-b7a8-f4120b31a45d nodeName:}" failed. No retries permitted until 2026-09-17 16:42:20.886155508 +0000 UTC m=+11.587234153 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f3d4b959-b664-439d-b7a8-f4120b31a45d-metrics-certs") pod "network-metrics-daemon-z77cp" (UID: "f3d4b959-b664-439d-b7a8-f4120b31a45d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 16:42:16.986396 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:16.986356    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/36250538-b6ed-4b8d-8788-a205a4dd53c6-original-pull-secret\") pod \"global-pull-secret-syncer-jpzxf\" (UID: \"36250538-b6ed-4b8d-8788-a205a4dd53c6\") " pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:16.986560 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:16.986454    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9t5z\" (UniqueName: \"kubernetes.io/projected/c3e3bba0-3038-411b-8194-5a88b7a0f443-kube-api-access-k9t5z\") pod \"network-check-target-nxc7t\" (UID: \"c3e3bba0-3038-411b-8194-5a88b7a0f443\") " pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:16.986637 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:16.986624    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 16:42:16.986685 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:16.986644    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 16:42:16.986685 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:16.986658    2586 projected.go:196] Error preparing data for projected volume kube-api-access-k9t5z for pod openshift-network-diagnostics/network-check-target-nxc7t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 16:42:16.986787 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:16.986715    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3e3bba0-3038-411b-8194-5a88b7a0f443-kube-api-access-k9t5z podName:c3e3bba0-3038-411b-8194-5a88b7a0f443 nodeName:}" failed. No retries permitted until 2026-09-17 16:42:20.986699231 +0000 UTC m=+11.687777873 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-k9t5z" (UniqueName: "kubernetes.io/projected/c3e3bba0-3038-411b-8194-5a88b7a0f443-kube-api-access-k9t5z") pod "network-check-target-nxc7t" (UID: "c3e3bba0-3038-411b-8194-5a88b7a0f443") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 16:42:16.987032 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:16.987006    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 16:42:16.987125 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:16.987076    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36250538-b6ed-4b8d-8788-a205a4dd53c6-original-pull-secret podName:36250538-b6ed-4b8d-8788-a205a4dd53c6 nodeName:}" failed. No retries permitted until 2026-09-17 16:42:20.987056785 +0000 UTC m=+11.688135429 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/36250538-b6ed-4b8d-8788-a205a4dd53c6-original-pull-secret") pod "global-pull-secret-syncer-jpzxf" (UID: "36250538-b6ed-4b8d-8788-a205a4dd53c6") : object "kube-system"/"original-pull-secret" not registered
Sep 17 16:42:17.321199 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:17.321123    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:17.321365 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:17.321259    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jpzxf" podUID="36250538-b6ed-4b8d-8788-a205a4dd53c6"
Sep 17 16:42:17.321651 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:17.321632    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:17.321749 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:17.321734    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z77cp" podUID="f3d4b959-b664-439d-b7a8-f4120b31a45d"
Sep 17 16:42:17.322714 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:17.322691    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:17.322828 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:17.322809    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nxc7t" podUID="c3e3bba0-3038-411b-8194-5a88b7a0f443"
Sep 17 16:42:19.319173 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:19.319139    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:19.319598 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:19.319184    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:19.319598 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:19.319268    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jpzxf" podUID="36250538-b6ed-4b8d-8788-a205a4dd53c6"
Sep 17 16:42:19.319598 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:19.319302    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:19.319598 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:19.319445    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z77cp" podUID="f3d4b959-b664-439d-b7a8-f4120b31a45d"
Sep 17 16:42:19.319598 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:19.319513    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nxc7t" podUID="c3e3bba0-3038-411b-8194-5a88b7a0f443"
Sep 17 16:42:20.921548 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:20.921505    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3d4b959-b664-439d-b7a8-f4120b31a45d-metrics-certs\") pod \"network-metrics-daemon-z77cp\" (UID: \"f3d4b959-b664-439d-b7a8-f4120b31a45d\") " pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:20.922030 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:20.921643    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 16:42:20.922030 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:20.921718    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d4b959-b664-439d-b7a8-f4120b31a45d-metrics-certs podName:f3d4b959-b664-439d-b7a8-f4120b31a45d nodeName:}" failed. No retries permitted until 2026-09-17 16:42:28.921698155 +0000 UTC m=+19.622776801 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f3d4b959-b664-439d-b7a8-f4120b31a45d-metrics-certs") pod "network-metrics-daemon-z77cp" (UID: "f3d4b959-b664-439d-b7a8-f4120b31a45d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 16:42:21.022203 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:21.022169    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9t5z\" (UniqueName: \"kubernetes.io/projected/c3e3bba0-3038-411b-8194-5a88b7a0f443-kube-api-access-k9t5z\") pod \"network-check-target-nxc7t\" (UID: \"c3e3bba0-3038-411b-8194-5a88b7a0f443\") " pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:21.022362 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:21.022258    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/36250538-b6ed-4b8d-8788-a205a4dd53c6-original-pull-secret\") pod \"global-pull-secret-syncer-jpzxf\" (UID: \"36250538-b6ed-4b8d-8788-a205a4dd53c6\") " pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:21.022404 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:21.022373    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 16:42:21.022434 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:21.022421    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36250538-b6ed-4b8d-8788-a205a4dd53c6-original-pull-secret podName:36250538-b6ed-4b8d-8788-a205a4dd53c6 nodeName:}" failed. No retries permitted until 2026-09-17 16:42:29.022409297 +0000 UTC m=+19.723487937 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/36250538-b6ed-4b8d-8788-a205a4dd53c6-original-pull-secret") pod "global-pull-secret-syncer-jpzxf" (UID: "36250538-b6ed-4b8d-8788-a205a4dd53c6") : object "kube-system"/"original-pull-secret" not registered
Sep 17 16:42:21.022787 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:21.022771    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 16:42:21.022838 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:21.022791    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 16:42:21.022838 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:21.022800    2586 projected.go:196] Error preparing data for projected volume kube-api-access-k9t5z for pod openshift-network-diagnostics/network-check-target-nxc7t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 16:42:21.022838 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:21.022835    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3e3bba0-3038-411b-8194-5a88b7a0f443-kube-api-access-k9t5z podName:c3e3bba0-3038-411b-8194-5a88b7a0f443 nodeName:}" failed. No retries permitted until 2026-09-17 16:42:29.022824032 +0000 UTC m=+19.723902686 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-k9t5z" (UniqueName: "kubernetes.io/projected/c3e3bba0-3038-411b-8194-5a88b7a0f443-kube-api-access-k9t5z") pod "network-check-target-nxc7t" (UID: "c3e3bba0-3038-411b-8194-5a88b7a0f443") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 16:42:21.320561 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:21.319724    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:21.320561 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:21.319839    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nxc7t" podUID="c3e3bba0-3038-411b-8194-5a88b7a0f443"
Sep 17 16:42:21.320561 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:21.320240    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:21.320561 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:21.320336    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jpzxf" podUID="36250538-b6ed-4b8d-8788-a205a4dd53c6"
Sep 17 16:42:21.320561 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:21.320394    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:21.320561 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:21.320472    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z77cp" podUID="f3d4b959-b664-439d-b7a8-f4120b31a45d"
Sep 17 16:42:23.319276 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:23.319246    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:23.319944 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:23.319376    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:23.319944 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:23.319384    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z77cp" podUID="f3d4b959-b664-439d-b7a8-f4120b31a45d"
Sep 17 16:42:23.319944 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:23.319494    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nxc7t" podUID="c3e3bba0-3038-411b-8194-5a88b7a0f443"
Sep 17 16:42:23.319944 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:23.319546    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:23.319944 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:23.319616    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jpzxf" podUID="36250538-b6ed-4b8d-8788-a205a4dd53c6"
Sep 17 16:42:25.319323 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:25.319279    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:25.319323 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:25.319321    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:25.319838 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:25.319279    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:25.319838 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:25.319427    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jpzxf" podUID="36250538-b6ed-4b8d-8788-a205a4dd53c6"
Sep 17 16:42:25.319838 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:25.319502    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nxc7t" podUID="c3e3bba0-3038-411b-8194-5a88b7a0f443"
Sep 17 16:42:25.319838 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:25.319614    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z77cp" podUID="f3d4b959-b664-439d-b7a8-f4120b31a45d"
Sep 17 16:42:27.318777 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:27.318743    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:27.319252 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:27.318743    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:27.319252 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:27.318852    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nxc7t" podUID="c3e3bba0-3038-411b-8194-5a88b7a0f443"
Sep 17 16:42:27.319252 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:27.318967    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jpzxf" podUID="36250538-b6ed-4b8d-8788-a205a4dd53c6"
Sep 17 16:42:27.319252 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:27.318753    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:27.319252 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:27.319099    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z77cp" podUID="f3d4b959-b664-439d-b7a8-f4120b31a45d"
Sep 17 16:42:28.981456 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:28.981419    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3d4b959-b664-439d-b7a8-f4120b31a45d-metrics-certs\") pod \"network-metrics-daemon-z77cp\" (UID: \"f3d4b959-b664-439d-b7a8-f4120b31a45d\") " pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:28.982060 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:28.981587    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 16:42:28.982060 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:28.981658    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d4b959-b664-439d-b7a8-f4120b31a45d-metrics-certs podName:f3d4b959-b664-439d-b7a8-f4120b31a45d nodeName:}" failed. No retries permitted until 2026-09-17 16:42:44.981643389 +0000 UTC m=+35.682722033 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f3d4b959-b664-439d-b7a8-f4120b31a45d-metrics-certs") pod "network-metrics-daemon-z77cp" (UID: "f3d4b959-b664-439d-b7a8-f4120b31a45d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 16:42:29.081857 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:29.081799    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9t5z\" (UniqueName: \"kubernetes.io/projected/c3e3bba0-3038-411b-8194-5a88b7a0f443-kube-api-access-k9t5z\") pod \"network-check-target-nxc7t\" (UID: \"c3e3bba0-3038-411b-8194-5a88b7a0f443\") " pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:29.082068 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:29.081911    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/36250538-b6ed-4b8d-8788-a205a4dd53c6-original-pull-secret\") pod \"global-pull-secret-syncer-jpzxf\" (UID: \"36250538-b6ed-4b8d-8788-a205a4dd53c6\") " pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:29.082068 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:29.082006    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 16:42:29.082068 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:29.082013    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 16:42:29.082068 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:29.082031    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 16:42:29.082068 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:29.082042    2586 projected.go:196] Error preparing data for projected volume kube-api-access-k9t5z for pod openshift-network-diagnostics/network-check-target-nxc7t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 16:42:29.082234 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:29.082076    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36250538-b6ed-4b8d-8788-a205a4dd53c6-original-pull-secret podName:36250538-b6ed-4b8d-8788-a205a4dd53c6 nodeName:}" failed. No retries permitted until 2026-09-17 16:42:45.082056865 +0000 UTC m=+35.783135508 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/36250538-b6ed-4b8d-8788-a205a4dd53c6-original-pull-secret") pod "global-pull-secret-syncer-jpzxf" (UID: "36250538-b6ed-4b8d-8788-a205a4dd53c6") : object "kube-system"/"original-pull-secret" not registered
Sep 17 16:42:29.082234 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:29.082089    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3e3bba0-3038-411b-8194-5a88b7a0f443-kube-api-access-k9t5z podName:c3e3bba0-3038-411b-8194-5a88b7a0f443 nodeName:}" failed. No retries permitted until 2026-09-17 16:42:45.082083301 +0000 UTC m=+35.783161941 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-k9t5z" (UniqueName: "kubernetes.io/projected/c3e3bba0-3038-411b-8194-5a88b7a0f443-kube-api-access-k9t5z") pod "network-check-target-nxc7t" (UID: "c3e3bba0-3038-411b-8194-5a88b7a0f443") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 16:42:29.319179 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:29.319054    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:29.319179 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:29.319093    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:29.319401 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:29.319067    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:29.319401 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:29.319192    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jpzxf" podUID="36250538-b6ed-4b8d-8788-a205a4dd53c6"
Sep 17 16:42:29.319401 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:29.319272    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nxc7t" podUID="c3e3bba0-3038-411b-8194-5a88b7a0f443"
Sep 17 16:42:29.319401 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:29.319356    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z77cp" podUID="f3d4b959-b664-439d-b7a8-f4120b31a45d"
Sep 17 16:42:31.319944 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:31.319905    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:31.319944 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:31.319940    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:31.320405 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:31.320031    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:31.320405 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:31.320040    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nxc7t" podUID="c3e3bba0-3038-411b-8194-5a88b7a0f443"
Sep 17 16:42:31.320405 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:31.320118    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z77cp" podUID="f3d4b959-b664-439d-b7a8-f4120b31a45d"
Sep 17 16:42:31.320405 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:31.320205    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jpzxf" podUID="36250538-b6ed-4b8d-8788-a205a4dd53c6"
Sep 17 16:42:32.388077 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:32.387755    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22fdn_68559fca-75ed-4ee8-bedd-57b04fbd3f4d/ovn-acl-logging/0.log"
Sep 17 16:42:32.388411 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:32.388384    2586 generic.go:356] "Generic (PLEG): container finished" podID="68559fca-75ed-4ee8-bedd-57b04fbd3f4d" containerID="b98409304eba68ed538e59ab099d1ac7231365ef4cd45684823672deef132278" exitCode=1
Sep 17 16:42:32.388479 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:32.388448    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-22fdn" event={"ID":"68559fca-75ed-4ee8-bedd-57b04fbd3f4d","Type":"ContainerStarted","Data":"b264f17515880892ee552253e55d674a0481cdd2d4e4715615dc581a8543dbc2"}
Sep 17 16:42:32.388580 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:32.388565    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-22fdn" event={"ID":"68559fca-75ed-4ee8-bedd-57b04fbd3f4d","Type":"ContainerStarted","Data":"aba534b4ef8c0df7452c9e20192ed6a9ba9d76dc2bb1c44d441e84ba0ce23d1a"}
Sep 17 16:42:32.388637 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:32.388621    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-22fdn" event={"ID":"68559fca-75ed-4ee8-bedd-57b04fbd3f4d","Type":"ContainerStarted","Data":"dfab086df124754f82702b1c4bc39959fba41c48801d8dce2cbfc96ff76abdce"}
Sep 17 16:42:32.388696 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:32.388671    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-22fdn" event={"ID":"68559fca-75ed-4ee8-bedd-57b04fbd3f4d","Type":"ContainerDied","Data":"b98409304eba68ed538e59ab099d1ac7231365ef4cd45684823672deef132278"}
Sep 17 16:42:32.388749 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:32.388722    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-22fdn" event={"ID":"68559fca-75ed-4ee8-bedd-57b04fbd3f4d","Type":"ContainerStarted","Data":"7f0d5ebff6c882b1eb2f8936fa0ae1604f1902e1929ad9389a6a03e759f1b85c"}
Sep 17 16:42:32.389831 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:32.389808    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-26cx9" event={"ID":"50ca1290-c198-4a93-9246-9707e5df2568","Type":"ContainerStarted","Data":"dde9b8aa650d4657b34fc1f55c7087b54aa2704d6af74037f97a96574ee7fb89"}
Sep 17 16:42:32.391296 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:32.391273    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-106.ec2.internal" event={"ID":"0052a71b7d9061ab5e61bebaf109be27","Type":"ContainerStarted","Data":"12a032f7906ecacbadae2a06dd483533331ff6b4939c45f1885cb714b2e03b9e"}
Sep 17 16:42:32.392624 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:32.392604    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hw7f7" event={"ID":"b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b","Type":"ContainerStarted","Data":"65a1b818e6c7fc931573e1ae3b642b5b6bee604a9384083b290e1f44ac3701ea"}
Sep 17 16:42:32.407408 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:32.407344    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-26cx9" podStartSLOduration=3.463194393 podStartE2EDuration="21.407329351s" totalImagesPullingTime="17.944134958s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 16:42:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:42:32.40726002 +0000 UTC m=+23.108338685" watchObservedRunningTime="2026-09-17 16:42:32.407329351 +0000 UTC m=+23.108408013"
Sep 17 16:42:32.438029 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:32.437981    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-106.ec2.internal" podStartSLOduration=20.437963855 podStartE2EDuration="20.437963855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 16:42:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:42:32.43785747 +0000 UTC m=+23.138936131" watchObservedRunningTime="2026-09-17 16:42:32.437963855 +0000 UTC m=+23.139042516"
Sep 17 16:42:32.438181 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:32.438097    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-hw7f7" podStartSLOduration=3.536107112 podStartE2EDuration="21.438090777s" totalImagesPullingTime="17.901983665s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 16:42:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:42:32.424261407 +0000 UTC m=+23.125340071" watchObservedRunningTime="2026-09-17 16:42:32.438090777 +0000 UTC m=+23.139169438"
Sep 17 16:42:33.319430 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:33.319400    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:33.319582 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:33.319399    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:33.319582 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:33.319514    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nxc7t" podUID="c3e3bba0-3038-411b-8194-5a88b7a0f443"
Sep 17 16:42:33.319582 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:33.319527    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:33.319700 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:33.319641    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jpzxf" podUID="36250538-b6ed-4b8d-8788-a205a4dd53c6"
Sep 17 16:42:33.319765 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:33.319747    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z77cp" podUID="f3d4b959-b664-439d-b7a8-f4120b31a45d"
Sep 17 16:42:33.400063 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:33.399770    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8flbg" event={"ID":"c77bcca8-72d5-49e4-89cc-c80e9492b22f","Type":"ContainerStarted","Data":"eeeedc292acdb82a204cf205f90f1c69950b962c61ba0bea2be57100a7b5c763"}
Sep 17 16:42:33.401795 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:33.401769    2586 generic.go:356] "Generic (PLEG): container finished" podID="b440e323-9b5d-4e31-b6e5-cd6c68d66451" containerID="5e2408b829ae952adbb681be756ff2c99eae6f75bd7fe0cb2adbdb5a1cdeedeb" exitCode=0
Sep 17 16:42:33.401949 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:33.401857    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cbv5m" event={"ID":"b440e323-9b5d-4e31-b6e5-cd6c68d66451","Type":"ContainerDied","Data":"5e2408b829ae952adbb681be756ff2c99eae6f75bd7fe0cb2adbdb5a1cdeedeb"}
Sep 17 16:42:33.403549 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:33.403525    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fktbq" event={"ID":"0fc32be3-c783-4e1b-83bc-45d67217b049","Type":"ContainerStarted","Data":"2b4ff64f649b36c4d8d16a464bd770fce8b5ce3b65892bf4ba69f66331bfac67"}
Sep 17 16:42:33.405481 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:33.405454    2586 generic.go:356] "Generic (PLEG): container finished" podID="882b812d92cabc3200681e9598ce9d95" containerID="800d0ddd216a4c77f224caf996fa1edf1036c7d4c74d87dcbccad4547ac52a24" exitCode=0
Sep 17 16:42:33.405588 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:33.405567    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-106.ec2.internal" event={"ID":"882b812d92cabc3200681e9598ce9d95","Type":"ContainerDied","Data":"800d0ddd216a4c77f224caf996fa1edf1036c7d4c74d87dcbccad4547ac52a24"}
Sep 17 16:42:33.407328 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:33.407304    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vsjlx" event={"ID":"7615eda5-5be1-4711-a3d6-e743d87c267a","Type":"ContainerStarted","Data":"d264556f752b66409923fb90d137181f036815b4a349eda2dbf7194be661f15a"}
Sep 17 16:42:33.409246 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:33.409219    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-h8gmd" event={"ID":"641beacd-a18d-42fe-a1f6-e75256b15892","Type":"ContainerStarted","Data":"de21dda1d94d0cf61166a6210d7ea739cfc08c04dadc4aec4845e7732c198fea"}
Sep 17 16:42:33.413115 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:33.413059    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22fdn_68559fca-75ed-4ee8-bedd-57b04fbd3f4d/ovn-acl-logging/0.log"
Sep 17 16:42:33.413573 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:33.413485    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-22fdn" event={"ID":"68559fca-75ed-4ee8-bedd-57b04fbd3f4d","Type":"ContainerStarted","Data":"1f0ff05d0e3bc903a7a69085770fa9c26950662ce2e5804867d630fe6fe636a2"}
Sep 17 16:42:33.415299 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:33.415276    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9rmt" event={"ID":"b37efedc-5602-4b17-af5e-01922a3a06dc","Type":"ContainerStarted","Data":"f514d5ead5c763720ffd30b14b1c12355e01e517dcccc4062787454401d4e075"}
Sep 17 16:42:33.415763 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:33.415724    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8flbg" podStartSLOduration=4.5254381299999995 podStartE2EDuration="22.415712546s" totalImagesPullingTime="17.890274416s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 16:42:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:42:33.415216574 +0000 UTC m=+24.116295236" watchObservedRunningTime="2026-09-17 16:42:33.415712546 +0000 UTC m=+24.116791209"
Sep 17 16:42:33.449051 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:33.448992    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-h8gmd" podStartSLOduration=4.576808371 podStartE2EDuration="22.448971311s" totalImagesPullingTime="17.87216294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:42:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:42:33.448919541 +0000 UTC m=+24.149998203" watchObservedRunningTime="2026-09-17 16:42:33.448971311 +0000 UTC m=+24.150049976"
Sep 17 16:42:33.471142 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:33.471089    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vsjlx" podStartSLOduration=4.402361823 podStartE2EDuration="22.471067363s" totalImagesPullingTime="18.06870554s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 16:42:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:42:33.470711855 +0000 UTC m=+24.171790517" watchObservedRunningTime="2026-09-17 16:42:33.471067363 +0000 UTC m=+24.172146024"
Sep 17 16:42:34.222255 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:34.222082    2586 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 17 16:42:34.331079 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:34.330887    2586 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-17T16:42:34.2222479Z","UUID":"67b6fa60-1763-4170-b7a1-053b5a29496b","Handler":null,"Name":"","Endpoint":""}
Sep 17 16:42:34.332575 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:34.332555    2586 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 17 16:42:34.332704 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:34.332585    2586 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 17 16:42:34.420506 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:34.420461    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-106.ec2.internal" event={"ID":"882b812d92cabc3200681e9598ce9d95","Type":"ContainerStarted","Data":"3eb8ebcfabf68c0455d14e07071611dd16e5315c5627f45758d316a5b68a83dd"}
Sep 17 16:42:34.422723 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:34.422694    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9rmt" event={"ID":"b37efedc-5602-4b17-af5e-01922a3a06dc","Type":"ContainerStarted","Data":"e8bd7c1c630f10f16846ead0e066d88284c515b373efb4ca37a613216661e067"}
Sep 17 16:42:34.435885 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:34.435828    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-fktbq" podStartSLOduration=5.527272324 podStartE2EDuration="23.435811277s" totalImagesPullingTime="17.908538953s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 16:42:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:42:33.508417195 +0000 UTC m=+24.209495857" watchObservedRunningTime="2026-09-17 16:42:34.435811277 +0000 UTC m=+25.136889939"
Sep 17 16:42:34.436257 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:34.436227    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-106.ec2.internal" podStartSLOduration=22.436219258 podStartE2EDuration="22.436219258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 16:42:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:42:34.435650676 +0000 UTC m=+25.136729337" watchObservedRunningTime="2026-09-17 16:42:34.436219258 +0000 UTC m=+25.137297911"
Sep 17 16:42:34.586684 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:34.586591    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-h8gmd"
Sep 17 16:42:34.587593 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:34.587316    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-h8gmd"
Sep 17 16:42:35.318989 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:35.318956    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:35.319165 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:35.318956    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:35.319165 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:35.318956    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:35.319285 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:35.319157    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z77cp" podUID="f3d4b959-b664-439d-b7a8-f4120b31a45d"
Sep 17 16:42:35.319285 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:35.319209    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jpzxf" podUID="36250538-b6ed-4b8d-8788-a205a4dd53c6"
Sep 17 16:42:35.319393 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:35.319298    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nxc7t" podUID="c3e3bba0-3038-411b-8194-5a88b7a0f443"
Sep 17 16:42:35.429702 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:35.429676    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22fdn_68559fca-75ed-4ee8-bedd-57b04fbd3f4d/ovn-acl-logging/0.log"
Sep 17 16:42:35.430379 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:35.430255    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-22fdn" event={"ID":"68559fca-75ed-4ee8-bedd-57b04fbd3f4d","Type":"ContainerStarted","Data":"6ae095bda4c79174fab6e92fb995d5993eecd420f15ca293f95e9cbb2ec1bdd9"}
Sep 17 16:42:35.733393 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:35.733310    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vsjlx_7615eda5-5be1-4711-a3d6-e743d87c267a/dns-node-resolver/0.log"
Sep 17 16:42:36.435070 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:36.435029    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9rmt" event={"ID":"b37efedc-5602-4b17-af5e-01922a3a06dc","Type":"ContainerStarted","Data":"106bbb338b92b1a6643d0a68f4bd80ac7322bc31388c4164b8c99b77a7f65d92"}
Sep 17 16:42:36.435070 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:36.435059    2586 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 17 16:42:36.454389 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:36.454333    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h9rmt" podStartSLOduration=5.327375428 podStartE2EDuration="25.454310447s" totalImagesPullingTime="20.126935019s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 16:42:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:42:36.453568571 +0000 UTC m=+27.154647234" watchObservedRunningTime="2026-09-17 16:42:36.454310447 +0000 UTC m=+27.155389111"
Sep 17 16:42:36.508547 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:36.508520    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fktbq_0fc32be3-c783-4e1b-83bc-45d67217b049/node-ca/0.log"
Sep 17 16:42:37.319010 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:37.318980    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:37.319183 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:37.318980    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:37.319183 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:37.319110    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:37.319183 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:37.319111    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nxc7t" podUID="c3e3bba0-3038-411b-8194-5a88b7a0f443"
Sep 17 16:42:37.319352 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:37.319209    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jpzxf" podUID="36250538-b6ed-4b8d-8788-a205a4dd53c6"
Sep 17 16:42:37.319352 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:37.319308    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z77cp" podUID="f3d4b959-b664-439d-b7a8-f4120b31a45d"
Sep 17 16:42:38.443521 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:38.443361    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22fdn_68559fca-75ed-4ee8-bedd-57b04fbd3f4d/ovn-acl-logging/0.log"
Sep 17 16:42:38.444021 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:38.443925    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-22fdn" event={"ID":"68559fca-75ed-4ee8-bedd-57b04fbd3f4d","Type":"ContainerStarted","Data":"3b5f171054b1bc69738940223165935b04d66cc443beda1d2441e2e9b5d4cc55"}
Sep 17 16:42:38.444277 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:38.444239    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:38.444395 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:38.444325    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:38.444395 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:38.444376    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:38.444395 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:38.444382    2586 scope.go:117] "RemoveContainer" containerID="b98409304eba68ed538e59ab099d1ac7231365ef4cd45684823672deef132278"
Sep 17 16:42:38.446018 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:38.445993    2586 generic.go:356] "Generic (PLEG): container finished" podID="b440e323-9b5d-4e31-b6e5-cd6c68d66451" containerID="fa39430fb60ebd69f89a39c03b08a794a040ee1c593ea402f7aa6e8c63c826d8" exitCode=0
Sep 17 16:42:38.446212 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:38.446030    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cbv5m" event={"ID":"b440e323-9b5d-4e31-b6e5-cd6c68d66451","Type":"ContainerDied","Data":"fa39430fb60ebd69f89a39c03b08a794a040ee1c593ea402f7aa6e8c63c826d8"}
Sep 17 16:42:38.461551 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:38.461528    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:38.461832 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:38.461815    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:42:38.735782 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:38.735687    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-h8gmd"
Sep 17 16:42:38.735954 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:38.735834    2586 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 17 16:42:38.736345 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:38.736320    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-h8gmd"
Sep 17 16:42:39.319498 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:39.319470    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:39.319671 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:39.319530    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:39.319671 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:39.319534    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:39.319671 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:39.319622    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jpzxf" podUID="36250538-b6ed-4b8d-8788-a205a4dd53c6"
Sep 17 16:42:39.319671 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:39.319660    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nxc7t" podUID="c3e3bba0-3038-411b-8194-5a88b7a0f443"
Sep 17 16:42:39.319848 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:39.319754    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z77cp" podUID="f3d4b959-b664-439d-b7a8-f4120b31a45d"
Sep 17 16:42:39.452395 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:39.452142    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22fdn_68559fca-75ed-4ee8-bedd-57b04fbd3f4d/ovn-acl-logging/0.log"
Sep 17 16:42:39.453148 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:39.452680    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-22fdn" event={"ID":"68559fca-75ed-4ee8-bedd-57b04fbd3f4d","Type":"ContainerStarted","Data":"9a97fd823d24dbf8cc93e24121f34eeaba6fba66a2651023b52243d1fd154321"}
Sep 17 16:42:39.454801 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:39.454778    2586 generic.go:356] "Generic (PLEG): container finished" podID="b440e323-9b5d-4e31-b6e5-cd6c68d66451" containerID="c4fc466fb035cc50beecd038dc5193048ace749b585bab20b901abd08f277e81" exitCode=0
Sep 17 16:42:39.454929 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:39.454861    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cbv5m" event={"ID":"b440e323-9b5d-4e31-b6e5-cd6c68d66451","Type":"ContainerDied","Data":"c4fc466fb035cc50beecd038dc5193048ace749b585bab20b901abd08f277e81"}
Sep 17 16:42:39.480068 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:39.480017    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-22fdn" podStartSLOduration=10.534287022000001 podStartE2EDuration="28.480003345s" totalImagesPullingTime="17.945716323s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 16:42:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:42:39.478455141 +0000 UTC m=+30.179533803" watchObservedRunningTime="2026-09-17 16:42:39.480003345 +0000 UTC m=+30.181082026"
Sep 17 16:42:39.575977 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:39.575937    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nxc7t"]
Sep 17 16:42:39.576156 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:39.576064    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:39.576220 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:39.576171    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nxc7t" podUID="c3e3bba0-3038-411b-8194-5a88b7a0f443"
Sep 17 16:42:39.578929 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:39.578901    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-z77cp"]
Sep 17 16:42:39.579070 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:39.579028    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:39.579160 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:39.579139    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z77cp" podUID="f3d4b959-b664-439d-b7a8-f4120b31a45d"
Sep 17 16:42:39.579597 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:39.579572    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jpzxf"]
Sep 17 16:42:39.579715 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:39.579674    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:39.579796 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:39.579770    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jpzxf" podUID="36250538-b6ed-4b8d-8788-a205a4dd53c6"
Sep 17 16:42:40.460253 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:40.460170    2586 generic.go:356] "Generic (PLEG): container finished" podID="b440e323-9b5d-4e31-b6e5-cd6c68d66451" containerID="55dc2fae5f80bfff1b1496ebd66db0cf28efc9c68a100fa480be7389dda00af9" exitCode=0
Sep 17 16:42:40.460610 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:40.460253    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cbv5m" event={"ID":"b440e323-9b5d-4e31-b6e5-cd6c68d66451","Type":"ContainerDied","Data":"55dc2fae5f80bfff1b1496ebd66db0cf28efc9c68a100fa480be7389dda00af9"}
Sep 17 16:42:41.320295 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:41.320268    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:41.320416 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:41.320378    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jpzxf" podUID="36250538-b6ed-4b8d-8788-a205a4dd53c6"
Sep 17 16:42:41.320564 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:41.320542    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:41.320691 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:41.320667    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z77cp" podUID="f3d4b959-b664-439d-b7a8-f4120b31a45d"
Sep 17 16:42:41.320759 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:41.320718    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:41.320825 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:41.320806    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nxc7t" podUID="c3e3bba0-3038-411b-8194-5a88b7a0f443"
Sep 17 16:42:43.318853 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:43.318810    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:43.318853 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:43.318859    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:43.319382 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:43.318810    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:43.319382 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:43.318970    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z77cp" podUID="f3d4b959-b664-439d-b7a8-f4120b31a45d"
Sep 17 16:42:43.319382 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:43.319100    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jpzxf" podUID="36250538-b6ed-4b8d-8788-a205a4dd53c6"
Sep 17 16:42:43.319382 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:43.319199    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nxc7t" podUID="c3e3bba0-3038-411b-8194-5a88b7a0f443"
Sep 17 16:42:45.006675 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:45.006576    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3d4b959-b664-439d-b7a8-f4120b31a45d-metrics-certs\") pod \"network-metrics-daemon-z77cp\" (UID: \"f3d4b959-b664-439d-b7a8-f4120b31a45d\") " pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:45.007174 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:45.006707    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 16:42:45.007174 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:45.006777    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d4b959-b664-439d-b7a8-f4120b31a45d-metrics-certs podName:f3d4b959-b664-439d-b7a8-f4120b31a45d nodeName:}" failed. No retries permitted until 2026-09-17 16:43:17.006758261 +0000 UTC m=+67.707836901 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f3d4b959-b664-439d-b7a8-f4120b31a45d-metrics-certs") pod "network-metrics-daemon-z77cp" (UID: "f3d4b959-b664-439d-b7a8-f4120b31a45d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 16:42:45.107524 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:45.107487    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9t5z\" (UniqueName: \"kubernetes.io/projected/c3e3bba0-3038-411b-8194-5a88b7a0f443-kube-api-access-k9t5z\") pod \"network-check-target-nxc7t\" (UID: \"c3e3bba0-3038-411b-8194-5a88b7a0f443\") " pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:45.107732 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:45.107540    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/36250538-b6ed-4b8d-8788-a205a4dd53c6-original-pull-secret\") pod \"global-pull-secret-syncer-jpzxf\" (UID: \"36250538-b6ed-4b8d-8788-a205a4dd53c6\") " pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:45.107732 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:45.107633    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 16:42:45.107732 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:45.107660    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 16:42:45.107732 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:45.107683    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36250538-b6ed-4b8d-8788-a205a4dd53c6-original-pull-secret podName:36250538-b6ed-4b8d-8788-a205a4dd53c6 nodeName:}" failed. No retries permitted until 2026-09-17 16:43:17.107670668 +0000 UTC m=+67.808749308 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/36250538-b6ed-4b8d-8788-a205a4dd53c6-original-pull-secret") pod "global-pull-secret-syncer-jpzxf" (UID: "36250538-b6ed-4b8d-8788-a205a4dd53c6") : object "kube-system"/"original-pull-secret" not registered
Sep 17 16:42:45.107732 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:45.107685    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 16:42:45.107732 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:45.107700    2586 projected.go:196] Error preparing data for projected volume kube-api-access-k9t5z for pod openshift-network-diagnostics/network-check-target-nxc7t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 16:42:45.107965 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:45.107746    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3e3bba0-3038-411b-8194-5a88b7a0f443-kube-api-access-k9t5z podName:c3e3bba0-3038-411b-8194-5a88b7a0f443 nodeName:}" failed. No retries permitted until 2026-09-17 16:43:17.107728495 +0000 UTC m=+67.808807134 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-k9t5z" (UniqueName: "kubernetes.io/projected/c3e3bba0-3038-411b-8194-5a88b7a0f443-kube-api-access-k9t5z") pod "network-check-target-nxc7t" (UID: "c3e3bba0-3038-411b-8194-5a88b7a0f443") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 16:42:45.319347 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:45.319313    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:45.319347 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:45.319350    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:45.319555 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:45.319313    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:45.319555 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:45.319428    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jpzxf" podUID="36250538-b6ed-4b8d-8788-a205a4dd53c6"
Sep 17 16:42:45.319555 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:45.319508    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z77cp" podUID="f3d4b959-b664-439d-b7a8-f4120b31a45d"
Sep 17 16:42:45.319655 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:45.319618    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nxc7t" podUID="c3e3bba0-3038-411b-8194-5a88b7a0f443"
Sep 17 16:42:45.476824 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:45.476782    2586 generic.go:356] "Generic (PLEG): container finished" podID="b440e323-9b5d-4e31-b6e5-cd6c68d66451" containerID="38f6eaf12bba4fc0514cf00867e9468df2b508adf234ead75bb5536264921d2d" exitCode=0
Sep 17 16:42:45.476999 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:45.476855    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cbv5m" event={"ID":"b440e323-9b5d-4e31-b6e5-cd6c68d66451","Type":"ContainerDied","Data":"38f6eaf12bba4fc0514cf00867e9468df2b508adf234ead75bb5536264921d2d"}
Sep 17 16:42:46.482810 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:46.482768    2586 generic.go:356] "Generic (PLEG): container finished" podID="b440e323-9b5d-4e31-b6e5-cd6c68d66451" containerID="43e7cb4b609ff0c743c96caa68e72849e37f05bac3e1284b8bccb71c7051499f" exitCode=0
Sep 17 16:42:46.483329 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:46.482815    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cbv5m" event={"ID":"b440e323-9b5d-4e31-b6e5-cd6c68d66451","Type":"ContainerDied","Data":"43e7cb4b609ff0c743c96caa68e72849e37f05bac3e1284b8bccb71c7051499f"}
Sep 17 16:42:47.319364 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:47.319156    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:47.319554 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:47.319153    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:47.319554 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:47.319461    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jpzxf" podUID="36250538-b6ed-4b8d-8788-a205a4dd53c6"
Sep 17 16:42:47.319554 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:47.319192    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:47.319554 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:47.319532    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nxc7t" podUID="c3e3bba0-3038-411b-8194-5a88b7a0f443"
Sep 17 16:42:47.319714 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:47.319634    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z77cp" podUID="f3d4b959-b664-439d-b7a8-f4120b31a45d"
Sep 17 16:42:47.489232 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:47.489200    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cbv5m" event={"ID":"b440e323-9b5d-4e31-b6e5-cd6c68d66451","Type":"ContainerStarted","Data":"e839436c18bf2281741d95709ef9ddf54bbd2085e997db0df2fc222751a8c65c"}
Sep 17 16:42:47.513996 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:47.513048    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-cbv5m" podStartSLOduration=8.767698587 podStartE2EDuration="36.513020405s" totalImagesPullingTime="27.745321818s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 16:42:11 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:42:47.511246261 +0000 UTC m=+38.212324922" watchObservedRunningTime="2026-09-17 16:42:47.513020405 +0000 UTC m=+38.214099073"
Sep 17 16:42:49.319456 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:49.319426    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:49.319993 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:49.319432    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:49.319993 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:49.319543    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nxc7t" podUID="c3e3bba0-3038-411b-8194-5a88b7a0f443"
Sep 17 16:42:49.319993 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:49.319618    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jpzxf" podUID="36250538-b6ed-4b8d-8788-a205a4dd53c6"
Sep 17 16:42:49.319993 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:49.319658    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:49.319993 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:49.319800    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z77cp" podUID="f3d4b959-b664-439d-b7a8-f4120b31a45d"
Sep 17 16:42:51.320146 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:51.320110    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:51.320641 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:51.320207    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:51.320641 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:51.320241    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:51.320641 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:51.320358    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z77cp" podUID="f3d4b959-b664-439d-b7a8-f4120b31a45d"
Sep 17 16:42:51.320641 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:51.320374    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nxc7t" podUID="c3e3bba0-3038-411b-8194-5a88b7a0f443"
Sep 17 16:42:51.320641 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:51.320447    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jpzxf" podUID="36250538-b6ed-4b8d-8788-a205a4dd53c6"
Sep 17 16:42:53.318740 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:53.318701    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:53.318740 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:53.318733    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:53.319258 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:53.318700    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:53.319258 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:53.318834    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nxc7t" podUID="c3e3bba0-3038-411b-8194-5a88b7a0f443"
Sep 17 16:42:53.319258 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:53.318927    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z77cp" podUID="f3d4b959-b664-439d-b7a8-f4120b31a45d"
Sep 17 16:42:53.319258 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:53.318979    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jpzxf" podUID="36250538-b6ed-4b8d-8788-a205a4dd53c6"
Sep 17 16:42:55.319424 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:55.319387    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:55.319993 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:55.319393    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:55.319993 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:55.319503    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jpzxf" podUID="36250538-b6ed-4b8d-8788-a205a4dd53c6"
Sep 17 16:42:55.319993 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:55.319604    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z77cp" podUID="f3d4b959-b664-439d-b7a8-f4120b31a45d"
Sep 17 16:42:55.319993 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:55.319407    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:55.319993 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:55.319673    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nxc7t" podUID="c3e3bba0-3038-411b-8194-5a88b7a0f443"
Sep 17 16:42:57.319430 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:57.319388    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:57.319938 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:57.319401    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:57.319938 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:57.319489    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jpzxf" podUID="36250538-b6ed-4b8d-8788-a205a4dd53c6"
Sep 17 16:42:57.319938 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:57.319578    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z77cp" podUID="f3d4b959-b664-439d-b7a8-f4120b31a45d"
Sep 17 16:42:57.319938 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:57.319631    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:57.319938 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:57.319685    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nxc7t" podUID="c3e3bba0-3038-411b-8194-5a88b7a0f443"
Sep 17 16:42:59.319487 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.319448    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:42:59.319487 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.319471    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:42:59.319953 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.319449    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:42:59.319953 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:59.319556    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jpzxf" podUID="36250538-b6ed-4b8d-8788-a205a4dd53c6"
Sep 17 16:42:59.319953 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:59.319625    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nxc7t" podUID="c3e3bba0-3038-411b-8194-5a88b7a0f443"
Sep 17 16:42:59.319953 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:42:59.319713    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z77cp" podUID="f3d4b959-b664-439d-b7a8-f4120b31a45d"
Sep 17 16:42:59.722031 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.721999    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-106.ec2.internal" event="NodeReady"
Sep 17 16:42:59.722228 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.722150    2586 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 17 16:42:59.763438 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.763409    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6b849696d6-2lzx6"]
Sep 17 16:42:59.787186 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.787148    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b849696d6-2lzx6"]
Sep 17 16:42:59.787357 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.787284    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b849696d6-2lzx6"
Sep 17 16:42:59.787852 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.787667    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wn8js"]
Sep 17 16:42:59.790308 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.790289    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 17 16:42:59.790476 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.790456    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rkgdc\""
Sep 17 16:42:59.790786 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.790768    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 17 16:42:59.791000 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.790984    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 17 16:42:59.796183 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.796161    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 17 16:42:59.802805 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.802783    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pnvqx"]
Sep 17 16:42:59.802936 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.802918    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wn8js"
Sep 17 16:42:59.805735 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.805714    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tcspl\""
Sep 17 16:42:59.805892 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.805853    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 17 16:42:59.806223 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.806204    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 17 16:42:59.806452 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.806269    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 17 16:42:59.823327 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.823268    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wn8js"]
Sep 17 16:42:59.823327 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.823302    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pnvqx"]
Sep 17 16:42:59.823505 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.823329    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pnvqx"
Sep 17 16:42:59.828268 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.828244    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 17 16:42:59.828389 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.828317    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-v5qhp\""
Sep 17 16:42:59.828449 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.828436    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 17 16:42:59.901181 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.901145    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zf89n"]
Sep 17 16:42:59.906565 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.906540    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5976328d-6d8e-4fcd-a23b-c1f229f2713f-tmp-dir\") pod \"dns-default-pnvqx\" (UID: \"5976328d-6d8e-4fcd-a23b-c1f229f2713f\") " pod="openshift-dns/dns-default-pnvqx"
Sep 17 16:42:59.906650 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.906575    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-725bc\" (UniqueName: \"kubernetes.io/projected/5976328d-6d8e-4fcd-a23b-c1f229f2713f-kube-api-access-725bc\") pod \"dns-default-pnvqx\" (UID: \"5976328d-6d8e-4fcd-a23b-c1f229f2713f\") " pod="openshift-dns/dns-default-pnvqx"
Sep 17 16:42:59.906650 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.906598    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/38641319-d38c-4818-a3dd-538d20dec869-cert\") pod \"ingress-canary-wn8js\" (UID: \"38641319-d38c-4818-a3dd-538d20dec869\") " pod="openshift-ingress-canary/ingress-canary-wn8js"
Sep 17 16:42:59.906650 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.906621    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e5484fbc-acb2-4ac6-ad26-0c7170c87ae5-registry-tls\") pod \"image-registry-6b849696d6-2lzx6\" (UID: \"e5484fbc-acb2-4ac6-ad26-0c7170c87ae5\") " pod="openshift-image-registry/image-registry-6b849696d6-2lzx6"
Sep 17 16:42:59.906768 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.906688    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5976328d-6d8e-4fcd-a23b-c1f229f2713f-metrics-tls\") pod \"dns-default-pnvqx\" (UID: \"5976328d-6d8e-4fcd-a23b-c1f229f2713f\") " pod="openshift-dns/dns-default-pnvqx"
Sep 17 16:42:59.906768 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.906730    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9gkz\" (UniqueName: \"kubernetes.io/projected/38641319-d38c-4818-a3dd-538d20dec869-kube-api-access-w9gkz\") pod \"ingress-canary-wn8js\" (UID: \"38641319-d38c-4818-a3dd-538d20dec869\") " pod="openshift-ingress-canary/ingress-canary-wn8js"
Sep 17 16:42:59.906768 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.906747    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5976328d-6d8e-4fcd-a23b-c1f229f2713f-config-volume\") pod \"dns-default-pnvqx\" (UID: \"5976328d-6d8e-4fcd-a23b-c1f229f2713f\") " pod="openshift-dns/dns-default-pnvqx"
Sep 17 16:42:59.906940 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.906773    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e5484fbc-acb2-4ac6-ad26-0c7170c87ae5-ca-trust-extracted\") pod \"image-registry-6b849696d6-2lzx6\" (UID: \"e5484fbc-acb2-4ac6-ad26-0c7170c87ae5\") " pod="openshift-image-registry/image-registry-6b849696d6-2lzx6"
Sep 17 16:42:59.906940 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.906862    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e5484fbc-acb2-4ac6-ad26-0c7170c87ae5-installation-pull-secrets\") pod \"image-registry-6b849696d6-2lzx6\" (UID: \"e5484fbc-acb2-4ac6-ad26-0c7170c87ae5\") " pod="openshift-image-registry/image-registry-6b849696d6-2lzx6"
Sep 17 16:42:59.907042 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.906932    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e5484fbc-acb2-4ac6-ad26-0c7170c87ae5-image-registry-private-configuration\") pod \"image-registry-6b849696d6-2lzx6\" (UID: \"e5484fbc-acb2-4ac6-ad26-0c7170c87ae5\") " pod="openshift-image-registry/image-registry-6b849696d6-2lzx6"
Sep 17 16:42:59.907042 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.906963    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flxvd\" (UniqueName: \"kubernetes.io/projected/e5484fbc-acb2-4ac6-ad26-0c7170c87ae5-kube-api-access-flxvd\") pod \"image-registry-6b849696d6-2lzx6\" (UID: \"e5484fbc-acb2-4ac6-ad26-0c7170c87ae5\") " pod="openshift-image-registry/image-registry-6b849696d6-2lzx6"
Sep 17 16:42:59.907042 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.906996    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e5484fbc-acb2-4ac6-ad26-0c7170c87ae5-registry-certificates\") pod \"image-registry-6b849696d6-2lzx6\" (UID: \"e5484fbc-acb2-4ac6-ad26-0c7170c87ae5\") " pod="openshift-image-registry/image-registry-6b849696d6-2lzx6"
Sep 17 16:42:59.907042 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.907035    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e5484fbc-acb2-4ac6-ad26-0c7170c87ae5-trusted-ca\") pod \"image-registry-6b849696d6-2lzx6\" (UID: \"e5484fbc-acb2-4ac6-ad26-0c7170c87ae5\") " pod="openshift-image-registry/image-registry-6b849696d6-2lzx6"
Sep 17 16:42:59.907187 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.907058    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e5484fbc-acb2-4ac6-ad26-0c7170c87ae5-bound-sa-token\") pod \"image-registry-6b849696d6-2lzx6\" (UID: \"e5484fbc-acb2-4ac6-ad26-0c7170c87ae5\") " pod="openshift-image-registry/image-registry-6b849696d6-2lzx6"
Sep 17 16:42:59.916805 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.916770    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zf89n"]
Sep 17 16:42:59.916948 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.916910    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zf89n"
Sep 17 16:42:59.919427 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.919402    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 17 16:42:59.919564 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.919507    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 17 16:42:59.919720 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.919701    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 17 16:42:59.919798 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.919702    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-d57nv\""
Sep 17 16:42:59.919856 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:42:59.919824    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 17 16:43:00.007639 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.007602    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e5484fbc-acb2-4ac6-ad26-0c7170c87ae5-registry-certificates\") pod \"image-registry-6b849696d6-2lzx6\" (UID: \"e5484fbc-acb2-4ac6-ad26-0c7170c87ae5\") " pod="openshift-image-registry/image-registry-6b849696d6-2lzx6"
Sep 17 16:43:00.007639 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.007639    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e5484fbc-acb2-4ac6-ad26-0c7170c87ae5-trusted-ca\") pod \"image-registry-6b849696d6-2lzx6\" (UID: \"e5484fbc-acb2-4ac6-ad26-0c7170c87ae5\") " pod="openshift-image-registry/image-registry-6b849696d6-2lzx6"
Sep 17 16:43:00.007864 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.007655    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e5484fbc-acb2-4ac6-ad26-0c7170c87ae5-bound-sa-token\") pod \"image-registry-6b849696d6-2lzx6\" (UID: \"e5484fbc-acb2-4ac6-ad26-0c7170c87ae5\") " pod="openshift-image-registry/image-registry-6b849696d6-2lzx6"
Sep 17 16:43:00.007941 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.007848    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fscw\" (UniqueName: \"kubernetes.io/projected/ae5edc41-0024-4ff8-825f-3fc6e3d20d08-kube-api-access-4fscw\") pod \"insights-runtime-extractor-zf89n\" (UID: \"ae5edc41-0024-4ff8-825f-3fc6e3d20d08\") " pod="openshift-insights/insights-runtime-extractor-zf89n"
Sep 17 16:43:00.007978 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.007935    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5976328d-6d8e-4fcd-a23b-c1f229f2713f-tmp-dir\") pod \"dns-default-pnvqx\" (UID: \"5976328d-6d8e-4fcd-a23b-c1f229f2713f\") " pod="openshift-dns/dns-default-pnvqx"
Sep 17 16:43:00.008022 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.007970    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-725bc\" (UniqueName: \"kubernetes.io/projected/5976328d-6d8e-4fcd-a23b-c1f229f2713f-kube-api-access-725bc\") pod \"dns-default-pnvqx\" (UID: \"5976328d-6d8e-4fcd-a23b-c1f229f2713f\") " pod="openshift-dns/dns-default-pnvqx"
Sep 17 16:43:00.008022 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.008007    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/38641319-d38c-4818-a3dd-538d20dec869-cert\") pod \"ingress-canary-wn8js\" (UID: \"38641319-d38c-4818-a3dd-538d20dec869\") " pod="openshift-ingress-canary/ingress-canary-wn8js"
Sep 17 16:43:00.008096 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.008040    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e5484fbc-acb2-4ac6-ad26-0c7170c87ae5-registry-tls\") pod \"image-registry-6b849696d6-2lzx6\" (UID: \"e5484fbc-acb2-4ac6-ad26-0c7170c87ae5\") " pod="openshift-image-registry/image-registry-6b849696d6-2lzx6"
Sep 17 16:43:00.008096 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.008091    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5976328d-6d8e-4fcd-a23b-c1f229f2713f-metrics-tls\") pod \"dns-default-pnvqx\" (UID: \"5976328d-6d8e-4fcd-a23b-c1f229f2713f\") " pod="openshift-dns/dns-default-pnvqx"
Sep 17 16:43:00.008197 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.008120    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ae5edc41-0024-4ff8-825f-3fc6e3d20d08-crio-socket\") pod \"insights-runtime-extractor-zf89n\" (UID: \"ae5edc41-0024-4ff8-825f-3fc6e3d20d08\") " pod="openshift-insights/insights-runtime-extractor-zf89n"
Sep 17 16:43:00.008197 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.008147    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9gkz\" (UniqueName: \"kubernetes.io/projected/38641319-d38c-4818-a3dd-538d20dec869-kube-api-access-w9gkz\") pod \"ingress-canary-wn8js\" (UID: \"38641319-d38c-4818-a3dd-538d20dec869\") " pod="openshift-ingress-canary/ingress-canary-wn8js"
Sep 17 16:43:00.008197 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.008171    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5976328d-6d8e-4fcd-a23b-c1f229f2713f-config-volume\") pod \"dns-default-pnvqx\" (UID: \"5976328d-6d8e-4fcd-a23b-c1f229f2713f\") " pod="openshift-dns/dns-default-pnvqx"
Sep 17 16:43:00.008333 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.008194    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ae5edc41-0024-4ff8-825f-3fc6e3d20d08-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zf89n\" (UID: \"ae5edc41-0024-4ff8-825f-3fc6e3d20d08\") " pod="openshift-insights/insights-runtime-extractor-zf89n"
Sep 17 16:43:00.008333 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.008239    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e5484fbc-acb2-4ac6-ad26-0c7170c87ae5-ca-trust-extracted\") pod \"image-registry-6b849696d6-2lzx6\" (UID: \"e5484fbc-acb2-4ac6-ad26-0c7170c87ae5\") " pod="openshift-image-registry/image-registry-6b849696d6-2lzx6"
Sep 17 16:43:00.008333 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.008261    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ae5edc41-0024-4ff8-825f-3fc6e3d20d08-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zf89n\" (UID: \"ae5edc41-0024-4ff8-825f-3fc6e3d20d08\") " pod="openshift-insights/insights-runtime-extractor-zf89n"
Sep 17 16:43:00.008333 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.008301    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e5484fbc-acb2-4ac6-ad26-0c7170c87ae5-installation-pull-secrets\") pod \"image-registry-6b849696d6-2lzx6\" (UID: \"e5484fbc-acb2-4ac6-ad26-0c7170c87ae5\") " pod="openshift-image-registry/image-registry-6b849696d6-2lzx6"
Sep 17 16:43:00.008502 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.008333    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e5484fbc-acb2-4ac6-ad26-0c7170c87ae5-image-registry-private-configuration\") pod \"image-registry-6b849696d6-2lzx6\" (UID: \"e5484fbc-acb2-4ac6-ad26-0c7170c87ae5\") " pod="openshift-image-registry/image-registry-6b849696d6-2lzx6"
Sep 17 16:43:00.008502 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.008360    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flxvd\" (UniqueName: \"kubernetes.io/projected/e5484fbc-acb2-4ac6-ad26-0c7170c87ae5-kube-api-access-flxvd\") pod \"image-registry-6b849696d6-2lzx6\" (UID: \"e5484fbc-acb2-4ac6-ad26-0c7170c87ae5\") " pod="openshift-image-registry/image-registry-6b849696d6-2lzx6"
Sep 17 16:43:00.008502 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.008383    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae5edc41-0024-4ff8-825f-3fc6e3d20d08-data-volume\") pod \"insights-runtime-extractor-zf89n\" (UID: \"ae5edc41-0024-4ff8-825f-3fc6e3d20d08\") " pod="openshift-insights/insights-runtime-extractor-zf89n"
Sep 17 16:43:00.008502 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.008397    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5976328d-6d8e-4fcd-a23b-c1f229f2713f-tmp-dir\") pod \"dns-default-pnvqx\" (UID: \"5976328d-6d8e-4fcd-a23b-c1f229f2713f\") " pod="openshift-dns/dns-default-pnvqx"
Sep 17 16:43:00.008671 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.008622    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e5484fbc-acb2-4ac6-ad26-0c7170c87ae5-registry-certificates\") pod \"image-registry-6b849696d6-2lzx6\" (UID: \"e5484fbc-acb2-4ac6-ad26-0c7170c87ae5\") " pod="openshift-image-registry/image-registry-6b849696d6-2lzx6"
Sep 17 16:43:00.008810 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.008766    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e5484fbc-acb2-4ac6-ad26-0c7170c87ae5-ca-trust-extracted\") pod \"image-registry-6b849696d6-2lzx6\" (UID: \"e5484fbc-acb2-4ac6-ad26-0c7170c87ae5\") " pod="openshift-image-registry/image-registry-6b849696d6-2lzx6"
Sep 17 16:43:00.008952 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.008923    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5976328d-6d8e-4fcd-a23b-c1f229f2713f-config-volume\") pod \"dns-default-pnvqx\" (UID: \"5976328d-6d8e-4fcd-a23b-c1f229f2713f\") " pod="openshift-dns/dns-default-pnvqx"
Sep 17 16:43:00.010164 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.010005    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e5484fbc-acb2-4ac6-ad26-0c7170c87ae5-trusted-ca\") pod \"image-registry-6b849696d6-2lzx6\" (UID: \"e5484fbc-acb2-4ac6-ad26-0c7170c87ae5\") " pod="openshift-image-registry/image-registry-6b849696d6-2lzx6"
Sep 17 16:43:00.012639 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.012608    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5976328d-6d8e-4fcd-a23b-c1f229f2713f-metrics-tls\") pod \"dns-default-pnvqx\" (UID: \"5976328d-6d8e-4fcd-a23b-c1f229f2713f\") " pod="openshift-dns/dns-default-pnvqx"
Sep 17 16:43:00.012639 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.012627    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e5484fbc-acb2-4ac6-ad26-0c7170c87ae5-installation-pull-secrets\") pod \"image-registry-6b849696d6-2lzx6\" (UID: \"e5484fbc-acb2-4ac6-ad26-0c7170c87ae5\") " pod="openshift-image-registry/image-registry-6b849696d6-2lzx6"
Sep 17 16:43:00.012765 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.012641    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/38641319-d38c-4818-a3dd-538d20dec869-cert\") pod \"ingress-canary-wn8js\" (UID: \"38641319-d38c-4818-a3dd-538d20dec869\") " pod="openshift-ingress-canary/ingress-canary-wn8js"
Sep 17 16:43:00.012765 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.012654    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e5484fbc-acb2-4ac6-ad26-0c7170c87ae5-registry-tls\") pod \"image-registry-6b849696d6-2lzx6\" (UID: \"e5484fbc-acb2-4ac6-ad26-0c7170c87ae5\") " pod="openshift-image-registry/image-registry-6b849696d6-2lzx6"
Sep 17 16:43:00.012972 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.012957    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e5484fbc-acb2-4ac6-ad26-0c7170c87ae5-image-registry-private-configuration\") pod \"image-registry-6b849696d6-2lzx6\" (UID: \"e5484fbc-acb2-4ac6-ad26-0c7170c87ae5\") " pod="openshift-image-registry/image-registry-6b849696d6-2lzx6"
Sep 17 16:43:00.016774 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.016745    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9gkz\" (UniqueName: \"kubernetes.io/projected/38641319-d38c-4818-a3dd-538d20dec869-kube-api-access-w9gkz\") pod \"ingress-canary-wn8js\" (UID: \"38641319-d38c-4818-a3dd-538d20dec869\") " pod="openshift-ingress-canary/ingress-canary-wn8js"
Sep 17 16:43:00.016904 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.016868    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-725bc\" (UniqueName: \"kubernetes.io/projected/5976328d-6d8e-4fcd-a23b-c1f229f2713f-kube-api-access-725bc\") pod \"dns-default-pnvqx\" (UID: \"5976328d-6d8e-4fcd-a23b-c1f229f2713f\") " pod="openshift-dns/dns-default-pnvqx"
Sep 17 16:43:00.017026 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.017011    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flxvd\" (UniqueName: \"kubernetes.io/projected/e5484fbc-acb2-4ac6-ad26-0c7170c87ae5-kube-api-access-flxvd\") pod \"image-registry-6b849696d6-2lzx6\" (UID: \"e5484fbc-acb2-4ac6-ad26-0c7170c87ae5\") " pod="openshift-image-registry/image-registry-6b849696d6-2lzx6"
Sep 17 16:43:00.026953 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.026932    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e5484fbc-acb2-4ac6-ad26-0c7170c87ae5-bound-sa-token\") pod \"image-registry-6b849696d6-2lzx6\" (UID: \"e5484fbc-acb2-4ac6-ad26-0c7170c87ae5\") " pod="openshift-image-registry/image-registry-6b849696d6-2lzx6"
Sep 17 16:43:00.097671 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.097584    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b849696d6-2lzx6"
Sep 17 16:43:00.108863 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.108832    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fscw\" (UniqueName: \"kubernetes.io/projected/ae5edc41-0024-4ff8-825f-3fc6e3d20d08-kube-api-access-4fscw\") pod \"insights-runtime-extractor-zf89n\" (UID: \"ae5edc41-0024-4ff8-825f-3fc6e3d20d08\") " pod="openshift-insights/insights-runtime-extractor-zf89n"
Sep 17 16:43:00.108958 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.108914    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ae5edc41-0024-4ff8-825f-3fc6e3d20d08-crio-socket\") pod \"insights-runtime-extractor-zf89n\" (UID: \"ae5edc41-0024-4ff8-825f-3fc6e3d20d08\") " pod="openshift-insights/insights-runtime-extractor-zf89n"
Sep 17 16:43:00.108958 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.108943    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ae5edc41-0024-4ff8-825f-3fc6e3d20d08-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zf89n\" (UID: \"ae5edc41-0024-4ff8-825f-3fc6e3d20d08\") " pod="openshift-insights/insights-runtime-extractor-zf89n"
Sep 17 16:43:00.109043 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.108973    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ae5edc41-0024-4ff8-825f-3fc6e3d20d08-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zf89n\" (UID: \"ae5edc41-0024-4ff8-825f-3fc6e3d20d08\") " pod="openshift-insights/insights-runtime-extractor-zf89n"
Sep 17 16:43:00.109125 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.109101    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae5edc41-0024-4ff8-825f-3fc6e3d20d08-data-volume\") pod \"insights-runtime-extractor-zf89n\" (UID: \"ae5edc41-0024-4ff8-825f-3fc6e3d20d08\") " pod="openshift-insights/insights-runtime-extractor-zf89n"
Sep 17 16:43:00.109183 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.109161    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ae5edc41-0024-4ff8-825f-3fc6e3d20d08-crio-socket\") pod \"insights-runtime-extractor-zf89n\" (UID: \"ae5edc41-0024-4ff8-825f-3fc6e3d20d08\") " pod="openshift-insights/insights-runtime-extractor-zf89n"
Sep 17 16:43:00.109381 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.109354    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae5edc41-0024-4ff8-825f-3fc6e3d20d08-data-volume\") pod \"insights-runtime-extractor-zf89n\" (UID: \"ae5edc41-0024-4ff8-825f-3fc6e3d20d08\") " pod="openshift-insights/insights-runtime-extractor-zf89n"
Sep 17 16:43:00.109553 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.109535    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ae5edc41-0024-4ff8-825f-3fc6e3d20d08-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zf89n\" (UID: \"ae5edc41-0024-4ff8-825f-3fc6e3d20d08\") " pod="openshift-insights/insights-runtime-extractor-zf89n"
Sep 17 16:43:00.111366 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.111345    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ae5edc41-0024-4ff8-825f-3fc6e3d20d08-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zf89n\" (UID: \"ae5edc41-0024-4ff8-825f-3fc6e3d20d08\") " pod="openshift-insights/insights-runtime-extractor-zf89n"
Sep 17 16:43:00.112242 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.112218    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wn8js"
Sep 17 16:43:00.117565 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.117547    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fscw\" (UniqueName: \"kubernetes.io/projected/ae5edc41-0024-4ff8-825f-3fc6e3d20d08-kube-api-access-4fscw\") pod \"insights-runtime-extractor-zf89n\" (UID: \"ae5edc41-0024-4ff8-825f-3fc6e3d20d08\") " pod="openshift-insights/insights-runtime-extractor-zf89n"
Sep 17 16:43:00.131465 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.131443    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pnvqx"
Sep 17 16:43:00.225851 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.225784    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zf89n"
Sep 17 16:43:00.434865 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.434765    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wn8js"]
Sep 17 16:43:00.439986 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.439959    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pnvqx"]
Sep 17 16:43:00.442797 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.442770    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b849696d6-2lzx6"]
Sep 17 16:43:00.444171 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:43:00.444142    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5976328d_6d8e_4fcd_a23b_c1f229f2713f.slice/crio-5c41aad032550848e07fb447bfb73178afa4311f20d71cd284a5ceb4aeb24350 WatchSource:0}: Error finding container 5c41aad032550848e07fb447bfb73178afa4311f20d71cd284a5ceb4aeb24350: Status 404 returned error can't find the container with id 5c41aad032550848e07fb447bfb73178afa4311f20d71cd284a5ceb4aeb24350
Sep 17 16:43:00.445465 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:43:00.445444    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5484fbc_acb2_4ac6_ad26_0c7170c87ae5.slice/crio-89eb78ea9664e14c14d01263fe3f6bf3f329614b2aff3b9db0f4868a8283af68 WatchSource:0}: Error finding container 89eb78ea9664e14c14d01263fe3f6bf3f329614b2aff3b9db0f4868a8283af68: Status 404 returned error can't find the container with id 89eb78ea9664e14c14d01263fe3f6bf3f329614b2aff3b9db0f4868a8283af68
Sep 17 16:43:00.518412 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.518378    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pnvqx" event={"ID":"5976328d-6d8e-4fcd-a23b-c1f229f2713f","Type":"ContainerStarted","Data":"5c41aad032550848e07fb447bfb73178afa4311f20d71cd284a5ceb4aeb24350"}
Sep 17 16:43:00.519580 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.519557    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wn8js" event={"ID":"38641319-d38c-4818-a3dd-538d20dec869","Type":"ContainerStarted","Data":"75997cf2e8c0bfbbf9c648bab4c56d071af8e4cb2781e7306c8e1e9c56e200e1"}
Sep 17 16:43:00.521038 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.521010    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b849696d6-2lzx6" event={"ID":"e5484fbc-acb2-4ac6-ad26-0c7170c87ae5","Type":"ContainerStarted","Data":"89eb78ea9664e14c14d01263fe3f6bf3f329614b2aff3b9db0f4868a8283af68"}
Sep 17 16:43:00.530644 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:00.530618    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zf89n"]
Sep 17 16:43:00.533810 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:43:00.533787    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae5edc41_0024_4ff8_825f_3fc6e3d20d08.slice/crio-108d0d7f94d9e41ea45e12e6b8c8b416384c0b8f2c174823a14e3b9ac9443234 WatchSource:0}: Error finding container 108d0d7f94d9e41ea45e12e6b8c8b416384c0b8f2c174823a14e3b9ac9443234: Status 404 returned error can't find the container with id 108d0d7f94d9e41ea45e12e6b8c8b416384c0b8f2c174823a14e3b9ac9443234
Sep 17 16:43:01.320451 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.320417    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:43:01.320637 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.320498    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:43:01.320637 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.320550    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:43:01.325142 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.325111    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kh76t\""
Sep 17 16:43:01.325272 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.325143    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 16:43:01.325537 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.325494    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ls2kp\""
Sep 17 16:43:01.325537 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.325518    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 16:43:01.325713 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.325557    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 16:43:01.325713 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.325612    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 16:43:01.525689 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.525654    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zf89n" event={"ID":"ae5edc41-0024-4ff8-825f-3fc6e3d20d08","Type":"ContainerStarted","Data":"a4faca48e8283b382e00143538e4d2983dd8abe6a1a1142f5a771913b46eea69"}
Sep 17 16:43:01.526095 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.525719    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zf89n" event={"ID":"ae5edc41-0024-4ff8-825f-3fc6e3d20d08","Type":"ContainerStarted","Data":"108d0d7f94d9e41ea45e12e6b8c8b416384c0b8f2c174823a14e3b9ac9443234"}
Sep 17 16:43:01.527777 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.527744    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b849696d6-2lzx6" event={"ID":"e5484fbc-acb2-4ac6-ad26-0c7170c87ae5","Type":"ContainerStarted","Data":"a5af34cd5c43d7936a6a96b1825849fc52a18e004dd331c7f66df9dbc4a193cb"}
Sep 17 16:43:01.527990 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.527958    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6b849696d6-2lzx6"
Sep 17 16:43:01.548953 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.548858    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6b849696d6-2lzx6" podStartSLOduration=2.548839414 podStartE2EDuration="2.548839414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:42:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:43:01.548316117 +0000 UTC m=+52.249394780" watchObservedRunningTime="2026-09-17 16:43:01.548839414 +0000 UTC m=+52.249918075"
Sep 17 16:43:01.916407 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.916349    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-fzbz8"]
Sep 17 16:43:01.938804 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.938764    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6db46dc76b-cnrnc"]
Sep 17 16:43:01.938990 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.938908    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fzbz8"
Sep 17 16:43:01.943094 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.942738    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 17 16:43:01.943094 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.942792    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 17 16:43:01.943094 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.942856    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 17 16:43:01.943094 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.942976    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-tvkpl\""
Sep 17 16:43:01.944088 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.944063    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 17 16:43:01.944534 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.944514    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 17 16:43:01.954497 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.954472    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-fzbz8"]
Sep 17 16:43:01.954663 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.954525    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6db46dc76b-cnrnc"]
Sep 17 16:43:01.954663 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.954619    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6db46dc76b-cnrnc"
Sep 17 16:43:01.957508 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.957482    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 17 16:43:01.957823 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.957806    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 17 16:43:01.958211 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.958192    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 17 16:43:01.958357 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.958341    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 17 16:43:01.958449 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.958424    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 17 16:43:01.958658 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.958606    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 17 16:43:01.958733 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.958691    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-qzqpx\""
Sep 17 16:43:01.958798 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:01.958631    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 17 16:43:02.128466 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.128432    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec225f41-f732-47ca-ac72-d92effb735b4-console-serving-cert\") pod \"console-6db46dc76b-cnrnc\" (UID: \"ec225f41-f732-47ca-ac72-d92effb735b4\") " pod="openshift-console/console-6db46dc76b-cnrnc"
Sep 17 16:43:02.128651 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.128502    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/93e836ed-cf52-4673-8258-ba74220df38b-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-fzbz8\" (UID: \"93e836ed-cf52-4673-8258-ba74220df38b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fzbz8"
Sep 17 16:43:02.128651 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.128535    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktjv6\" (UniqueName: \"kubernetes.io/projected/93e836ed-cf52-4673-8258-ba74220df38b-kube-api-access-ktjv6\") pod \"prometheus-operator-7bd5d56d6-fzbz8\" (UID: \"93e836ed-cf52-4673-8258-ba74220df38b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fzbz8"
Sep 17 16:43:02.128651 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.128563    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec225f41-f732-47ca-ac72-d92effb735b4-service-ca\") pod \"console-6db46dc76b-cnrnc\" (UID: \"ec225f41-f732-47ca-ac72-d92effb735b4\") " pod="openshift-console/console-6db46dc76b-cnrnc"
Sep 17 16:43:02.128651 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.128607    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec225f41-f732-47ca-ac72-d92effb735b4-console-oauth-config\") pod \"console-6db46dc76b-cnrnc\" (UID: \"ec225f41-f732-47ca-ac72-d92effb735b4\") " pod="openshift-console/console-6db46dc76b-cnrnc"
Sep 17 16:43:02.128834 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.128653    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec225f41-f732-47ca-ac72-d92effb735b4-console-config\") pod \"console-6db46dc76b-cnrnc\" (UID: \"ec225f41-f732-47ca-ac72-d92effb735b4\") " pod="openshift-console/console-6db46dc76b-cnrnc"
Sep 17 16:43:02.128834 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.128689    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93e836ed-cf52-4673-8258-ba74220df38b-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-fzbz8\" (UID: \"93e836ed-cf52-4673-8258-ba74220df38b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fzbz8"
Sep 17 16:43:02.128834 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.128716    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec225f41-f732-47ca-ac72-d92effb735b4-oauth-serving-cert\") pod \"console-6db46dc76b-cnrnc\" (UID: \"ec225f41-f732-47ca-ac72-d92effb735b4\") " pod="openshift-console/console-6db46dc76b-cnrnc"
Sep 17 16:43:02.128834 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.128757    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/93e836ed-cf52-4673-8258-ba74220df38b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-fzbz8\" (UID: \"93e836ed-cf52-4673-8258-ba74220df38b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fzbz8"
Sep 17 16:43:02.128834 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.128799    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlrb7\" (UniqueName: \"kubernetes.io/projected/ec225f41-f732-47ca-ac72-d92effb735b4-kube-api-access-vlrb7\") pod \"console-6db46dc76b-cnrnc\" (UID: \"ec225f41-f732-47ca-ac72-d92effb735b4\") " pod="openshift-console/console-6db46dc76b-cnrnc"
Sep 17 16:43:02.230219 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.230118    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec225f41-f732-47ca-ac72-d92effb735b4-console-serving-cert\") pod \"console-6db46dc76b-cnrnc\" (UID: \"ec225f41-f732-47ca-ac72-d92effb735b4\") " pod="openshift-console/console-6db46dc76b-cnrnc"
Sep 17 16:43:02.230219 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.230199    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/93e836ed-cf52-4673-8258-ba74220df38b-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-fzbz8\" (UID: \"93e836ed-cf52-4673-8258-ba74220df38b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fzbz8"
Sep 17 16:43:02.230429 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.230224    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktjv6\" (UniqueName: \"kubernetes.io/projected/93e836ed-cf52-4673-8258-ba74220df38b-kube-api-access-ktjv6\") pod \"prometheus-operator-7bd5d56d6-fzbz8\" (UID: \"93e836ed-cf52-4673-8258-ba74220df38b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fzbz8"
Sep 17 16:43:02.230429 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.230252    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec225f41-f732-47ca-ac72-d92effb735b4-service-ca\") pod \"console-6db46dc76b-cnrnc\" (UID: \"ec225f41-f732-47ca-ac72-d92effb735b4\") " pod="openshift-console/console-6db46dc76b-cnrnc"
Sep 17 16:43:02.230429 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.230381    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec225f41-f732-47ca-ac72-d92effb735b4-console-oauth-config\") pod \"console-6db46dc76b-cnrnc\" (UID: \"ec225f41-f732-47ca-ac72-d92effb735b4\") " pod="openshift-console/console-6db46dc76b-cnrnc"
Sep 17 16:43:02.230429 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.230419    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec225f41-f732-47ca-ac72-d92effb735b4-console-config\") pod \"console-6db46dc76b-cnrnc\" (UID: \"ec225f41-f732-47ca-ac72-d92effb735b4\") " pod="openshift-console/console-6db46dc76b-cnrnc"
Sep 17 16:43:02.230630 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.230445    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93e836ed-cf52-4673-8258-ba74220df38b-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-fzbz8\" (UID: \"93e836ed-cf52-4673-8258-ba74220df38b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fzbz8"
Sep 17 16:43:02.230630 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.230472    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec225f41-f732-47ca-ac72-d92effb735b4-oauth-serving-cert\") pod \"console-6db46dc76b-cnrnc\" (UID: \"ec225f41-f732-47ca-ac72-d92effb735b4\") " pod="openshift-console/console-6db46dc76b-cnrnc"
Sep 17 16:43:02.230630 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.230510    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/93e836ed-cf52-4673-8258-ba74220df38b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-fzbz8\" (UID: \"93e836ed-cf52-4673-8258-ba74220df38b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fzbz8"
Sep 17 16:43:02.230630 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.230598    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlrb7\" (UniqueName: \"kubernetes.io/projected/ec225f41-f732-47ca-ac72-d92effb735b4-kube-api-access-vlrb7\") pod \"console-6db46dc76b-cnrnc\" (UID: \"ec225f41-f732-47ca-ac72-d92effb735b4\") " pod="openshift-console/console-6db46dc76b-cnrnc"
Sep 17 16:43:02.231463 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.231359    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec225f41-f732-47ca-ac72-d92effb735b4-console-config\") pod \"console-6db46dc76b-cnrnc\" (UID: \"ec225f41-f732-47ca-ac72-d92effb735b4\") " pod="openshift-console/console-6db46dc76b-cnrnc"
Sep 17 16:43:02.231592 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.231511    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec225f41-f732-47ca-ac72-d92effb735b4-oauth-serving-cert\") pod \"console-6db46dc76b-cnrnc\" (UID: \"ec225f41-f732-47ca-ac72-d92effb735b4\") " pod="openshift-console/console-6db46dc76b-cnrnc"
Sep 17 16:43:02.231592 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.231557    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec225f41-f732-47ca-ac72-d92effb735b4-service-ca\") pod \"console-6db46dc76b-cnrnc\" (UID: \"ec225f41-f732-47ca-ac72-d92effb735b4\") " pod="openshift-console/console-6db46dc76b-cnrnc"
Sep 17 16:43:02.231804 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.231780    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93e836ed-cf52-4673-8258-ba74220df38b-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-fzbz8\" (UID: \"93e836ed-cf52-4673-8258-ba74220df38b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fzbz8"
Sep 17 16:43:02.233212 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.233188    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/93e836ed-cf52-4673-8258-ba74220df38b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-fzbz8\" (UID: \"93e836ed-cf52-4673-8258-ba74220df38b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fzbz8"
Sep 17 16:43:02.233338 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.233191    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec225f41-f732-47ca-ac72-d92effb735b4-console-serving-cert\") pod \"console-6db46dc76b-cnrnc\" (UID: \"ec225f41-f732-47ca-ac72-d92effb735b4\") " pod="openshift-console/console-6db46dc76b-cnrnc"
Sep 17 16:43:02.233338 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.233258    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec225f41-f732-47ca-ac72-d92effb735b4-console-oauth-config\") pod \"console-6db46dc76b-cnrnc\" (UID: \"ec225f41-f732-47ca-ac72-d92effb735b4\") " pod="openshift-console/console-6db46dc76b-cnrnc"
Sep 17 16:43:02.233338 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.233296    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/93e836ed-cf52-4673-8258-ba74220df38b-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-fzbz8\" (UID: \"93e836ed-cf52-4673-8258-ba74220df38b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fzbz8"
Sep 17 16:43:02.238972 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.238942    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktjv6\" (UniqueName: \"kubernetes.io/projected/93e836ed-cf52-4673-8258-ba74220df38b-kube-api-access-ktjv6\") pod \"prometheus-operator-7bd5d56d6-fzbz8\" (UID: \"93e836ed-cf52-4673-8258-ba74220df38b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fzbz8"
Sep 17 16:43:02.239525 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.239504    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlrb7\" (UniqueName: \"kubernetes.io/projected/ec225f41-f732-47ca-ac72-d92effb735b4-kube-api-access-vlrb7\") pod \"console-6db46dc76b-cnrnc\" (UID: \"ec225f41-f732-47ca-ac72-d92effb735b4\") " pod="openshift-console/console-6db46dc76b-cnrnc"
Sep 17 16:43:02.251428 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.251407    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fzbz8"
Sep 17 16:43:02.267660 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:02.267619    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6db46dc76b-cnrnc"
Sep 17 16:43:03.711015 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:03.710990    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-fzbz8"]
Sep 17 16:43:03.714083 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:03.714058    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6db46dc76b-cnrnc"]
Sep 17 16:43:03.714753 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:43:03.714730    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93e836ed_cf52_4673_8258_ba74220df38b.slice/crio-75cf03de479b001a2ade19f84f17774ae51e8bcad2293364878b196cf3abaf39 WatchSource:0}: Error finding container 75cf03de479b001a2ade19f84f17774ae51e8bcad2293364878b196cf3abaf39: Status 404 returned error can't find the container with id 75cf03de479b001a2ade19f84f17774ae51e8bcad2293364878b196cf3abaf39
Sep 17 16:43:03.717220 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:43:03.717197    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec225f41_f732_47ca_ac72_d92effb735b4.slice/crio-e9018fe0f5c197cfedb2f762e0872a290b255f2bd39c9b2822f170dd9750f43f WatchSource:0}: Error finding container e9018fe0f5c197cfedb2f762e0872a290b255f2bd39c9b2822f170dd9750f43f: Status 404 returned error can't find the container with id e9018fe0f5c197cfedb2f762e0872a290b255f2bd39c9b2822f170dd9750f43f
Sep 17 16:43:04.543573 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:04.543511    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fzbz8" event={"ID":"93e836ed-cf52-4673-8258-ba74220df38b","Type":"ContainerStarted","Data":"75cf03de479b001a2ade19f84f17774ae51e8bcad2293364878b196cf3abaf39"}
Sep 17 16:43:04.548166 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:04.548118    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6db46dc76b-cnrnc" event={"ID":"ec225f41-f732-47ca-ac72-d92effb735b4","Type":"ContainerStarted","Data":"e9018fe0f5c197cfedb2f762e0872a290b255f2bd39c9b2822f170dd9750f43f"}
Sep 17 16:43:04.563779 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:04.563738    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pnvqx" event={"ID":"5976328d-6d8e-4fcd-a23b-c1f229f2713f","Type":"ContainerStarted","Data":"59e95288d717cd907df003feca4384f664fc930c3ce36512800c6c138c6abc0b"}
Sep 17 16:43:04.563968 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:04.563812    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pnvqx" event={"ID":"5976328d-6d8e-4fcd-a23b-c1f229f2713f","Type":"ContainerStarted","Data":"eb32516319164717db55fe2bfbca75a604846b05c999f4e088905c40d787c49d"}
Sep 17 16:43:04.564245 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:04.564181    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pnvqx"
Sep 17 16:43:04.567384 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:04.567354    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zf89n" event={"ID":"ae5edc41-0024-4ff8-825f-3fc6e3d20d08","Type":"ContainerStarted","Data":"7e15442029e185b0ed00fc800b0d869eea305846a7d38ca5037b301e725c2344"}
Sep 17 16:43:04.571915 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:04.571492    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wn8js" event={"ID":"38641319-d38c-4818-a3dd-538d20dec869","Type":"ContainerStarted","Data":"f05431b45f468ad9b3c0c1942e01b894677febf36bde1bd49b22d2e0d775a359"}
Sep 17 16:43:04.601056 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:04.599968    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wn8js" podStartSLOduration=2.597861676 podStartE2EDuration="5.599945417s" totalImagesPullingTime="3.002083741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:42:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:43:04.599721835 +0000 UTC m=+55.300800497" watchObservedRunningTime="2026-09-17 16:43:04.599945417 +0000 UTC m=+55.301024096"
Sep 17 16:43:04.601056 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:04.600083    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-pnvqx" podStartSLOduration=2.611404503 podStartE2EDuration="5.600073583s" totalImagesPullingTime="2.98866908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:42:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:43:04.583077479 +0000 UTC m=+55.284156178" watchObservedRunningTime="2026-09-17 16:43:04.600073583 +0000 UTC m=+55.301152261"
Sep 17 16:43:08.588492 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:08.588390    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zf89n" event={"ID":"ae5edc41-0024-4ff8-825f-3fc6e3d20d08","Type":"ContainerStarted","Data":"202346740ae7516ac023dbdd39224752179e8e1ff706302f2312ddc6c35a4100"}
Sep 17 16:43:08.590393 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:08.590358    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fzbz8" event={"ID":"93e836ed-cf52-4673-8258-ba74220df38b","Type":"ContainerStarted","Data":"bc2091dbca934585fa52cec2d7446cc429aead079454f357aae98f3cf482e267"}
Sep 17 16:43:08.590523 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:08.590420    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fzbz8" event={"ID":"93e836ed-cf52-4673-8258-ba74220df38b","Type":"ContainerStarted","Data":"deefb9dd11ee677deee1f2b621bb3b1d31eeca269646589945d552430a51e926"}
Sep 17 16:43:08.592157 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:08.592129    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6db46dc76b-cnrnc" event={"ID":"ec225f41-f732-47ca-ac72-d92effb735b4","Type":"ContainerStarted","Data":"74a078a4f5170134876286edde1b1debf065e9ec46e66648504ce1e087fb996d"}
Sep 17 16:43:08.607651 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:08.607602    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zf89n" podStartSLOduration=2.10642268 podStartE2EDuration="9.607584674s" totalImagesPullingTime="7.501161994s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 16:42:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:43:08.606196625 +0000 UTC m=+59.307275283" watchObservedRunningTime="2026-09-17 16:43:08.607584674 +0000 UTC m=+59.308663362"
Sep 17 16:43:08.625360 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:08.625307    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6db46dc76b-cnrnc" podStartSLOduration=2.998953261 podStartE2EDuration="7.62529141s" totalImagesPullingTime="4.626338149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:43:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:43:08.624203573 +0000 UTC m=+59.325282237" watchObservedRunningTime="2026-09-17 16:43:08.62529141 +0000 UTC m=+59.326370136"
Sep 17 16:43:08.641253 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:08.641197    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-fzbz8" podStartSLOduration=3.056887532 podStartE2EDuration="7.641178811s" totalImagesPullingTime="4.584291279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:43:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:43:08.640817241 +0000 UTC m=+59.341895897" watchObservedRunningTime="2026-09-17 16:43:08.641178811 +0000 UTC m=+59.342257473"
Sep 17 16:43:10.275207 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.275170    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2k2gh"]
Sep 17 16:43:10.277381 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.277359    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2k2gh"
Sep 17 16:43:10.282366 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.282340    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wwxz2\""
Sep 17 16:43:10.282772 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.282755    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 17 16:43:10.283591 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.283287    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 17 16:43:10.285399 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.285379    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 17 16:43:10.291913 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.291889    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ae9d0d08-59de-454d-b7c5-45fc0393a7f3-sys\") pod \"node-exporter-2k2gh\" (UID: \"ae9d0d08-59de-454d-b7c5-45fc0393a7f3\") " pod="openshift-monitoring/node-exporter-2k2gh"
Sep 17 16:43:10.292010 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.291923    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ae9d0d08-59de-454d-b7c5-45fc0393a7f3-node-exporter-accelerators-collector-config\") pod \"node-exporter-2k2gh\" (UID: \"ae9d0d08-59de-454d-b7c5-45fc0393a7f3\") " pod="openshift-monitoring/node-exporter-2k2gh"
Sep 17 16:43:10.292010 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.291960    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae9d0d08-59de-454d-b7c5-45fc0393a7f3-metrics-client-ca\") pod \"node-exporter-2k2gh\" (UID: \"ae9d0d08-59de-454d-b7c5-45fc0393a7f3\") " pod="openshift-monitoring/node-exporter-2k2gh"
Sep 17 16:43:10.292010 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.291976    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae9d0d08-59de-454d-b7c5-45fc0393a7f3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2k2gh\" (UID: \"ae9d0d08-59de-454d-b7c5-45fc0393a7f3\") " pod="openshift-monitoring/node-exporter-2k2gh"
Sep 17 16:43:10.292010 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.291994    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ae9d0d08-59de-454d-b7c5-45fc0393a7f3-root\") pod \"node-exporter-2k2gh\" (UID: \"ae9d0d08-59de-454d-b7c5-45fc0393a7f3\") " pod="openshift-monitoring/node-exporter-2k2gh"
Sep 17 16:43:10.292225 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.292056    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ae9d0d08-59de-454d-b7c5-45fc0393a7f3-node-exporter-tls\") pod \"node-exporter-2k2gh\" (UID: \"ae9d0d08-59de-454d-b7c5-45fc0393a7f3\") " pod="openshift-monitoring/node-exporter-2k2gh"
Sep 17 16:43:10.292225 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.292097    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ae9d0d08-59de-454d-b7c5-45fc0393a7f3-node-exporter-wtmp\") pod \"node-exporter-2k2gh\" (UID: \"ae9d0d08-59de-454d-b7c5-45fc0393a7f3\") " pod="openshift-monitoring/node-exporter-2k2gh"
Sep 17 16:43:10.292225 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.292155    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv9bs\" (UniqueName: \"kubernetes.io/projected/ae9d0d08-59de-454d-b7c5-45fc0393a7f3-kube-api-access-fv9bs\") pod \"node-exporter-2k2gh\" (UID: \"ae9d0d08-59de-454d-b7c5-45fc0393a7f3\") " pod="openshift-monitoring/node-exporter-2k2gh"
Sep 17 16:43:10.292343 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.292235    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ae9d0d08-59de-454d-b7c5-45fc0393a7f3-node-exporter-textfile\") pod \"node-exporter-2k2gh\" (UID: \"ae9d0d08-59de-454d-b7c5-45fc0393a7f3\") " pod="openshift-monitoring/node-exporter-2k2gh"
Sep 17 16:43:10.392730 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.392691    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ae9d0d08-59de-454d-b7c5-45fc0393a7f3-node-exporter-textfile\") pod \"node-exporter-2k2gh\" (UID: \"ae9d0d08-59de-454d-b7c5-45fc0393a7f3\") " pod="openshift-monitoring/node-exporter-2k2gh"
Sep 17 16:43:10.392730 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.392733    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ae9d0d08-59de-454d-b7c5-45fc0393a7f3-sys\") pod \"node-exporter-2k2gh\" (UID: \"ae9d0d08-59de-454d-b7c5-45fc0393a7f3\") " pod="openshift-monitoring/node-exporter-2k2gh"
Sep 17 16:43:10.393041 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.392753    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ae9d0d08-59de-454d-b7c5-45fc0393a7f3-node-exporter-accelerators-collector-config\") pod \"node-exporter-2k2gh\" (UID: \"ae9d0d08-59de-454d-b7c5-45fc0393a7f3\") " pod="openshift-monitoring/node-exporter-2k2gh"
Sep 17 16:43:10.393041 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.392790    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae9d0d08-59de-454d-b7c5-45fc0393a7f3-metrics-client-ca\") pod \"node-exporter-2k2gh\" (UID: \"ae9d0d08-59de-454d-b7c5-45fc0393a7f3\") " pod="openshift-monitoring/node-exporter-2k2gh"
Sep 17 16:43:10.393041 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.392815    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae9d0d08-59de-454d-b7c5-45fc0393a7f3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2k2gh\" (UID: \"ae9d0d08-59de-454d-b7c5-45fc0393a7f3\") " pod="openshift-monitoring/node-exporter-2k2gh"
Sep 17 16:43:10.393041 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.392835    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ae9d0d08-59de-454d-b7c5-45fc0393a7f3-root\") pod \"node-exporter-2k2gh\" (UID: \"ae9d0d08-59de-454d-b7c5-45fc0393a7f3\") " pod="openshift-monitoring/node-exporter-2k2gh"
Sep 17 16:43:10.393041 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.392836    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ae9d0d08-59de-454d-b7c5-45fc0393a7f3-sys\") pod \"node-exporter-2k2gh\" (UID: \"ae9d0d08-59de-454d-b7c5-45fc0393a7f3\") " pod="openshift-monitoring/node-exporter-2k2gh"
Sep 17 16:43:10.393041 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.392861    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ae9d0d08-59de-454d-b7c5-45fc0393a7f3-node-exporter-tls\") pod \"node-exporter-2k2gh\" (UID: \"ae9d0d08-59de-454d-b7c5-45fc0393a7f3\") " pod="openshift-monitoring/node-exporter-2k2gh"
Sep 17 16:43:10.393041 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.392943    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ae9d0d08-59de-454d-b7c5-45fc0393a7f3-root\") pod \"node-exporter-2k2gh\" (UID: \"ae9d0d08-59de-454d-b7c5-45fc0393a7f3\") " pod="openshift-monitoring/node-exporter-2k2gh"
Sep 17 16:43:10.393041 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.392945    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ae9d0d08-59de-454d-b7c5-45fc0393a7f3-node-exporter-wtmp\") pod \"node-exporter-2k2gh\" (UID: \"ae9d0d08-59de-454d-b7c5-45fc0393a7f3\") " pod="openshift-monitoring/node-exporter-2k2gh"
Sep 17 16:43:10.393041 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.392992    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv9bs\" (UniqueName: \"kubernetes.io/projected/ae9d0d08-59de-454d-b7c5-45fc0393a7f3-kube-api-access-fv9bs\") pod \"node-exporter-2k2gh\" (UID: \"ae9d0d08-59de-454d-b7c5-45fc0393a7f3\") " pod="openshift-monitoring/node-exporter-2k2gh"
Sep 17 16:43:10.393041 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:43:10.393028    2586 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 17 16:43:10.393440 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:43:10.393096    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae9d0d08-59de-454d-b7c5-45fc0393a7f3-node-exporter-tls podName:ae9d0d08-59de-454d-b7c5-45fc0393a7f3 nodeName:}" failed. No retries permitted until 2026-09-17 16:43:10.8930764 +0000 UTC m=+61.594155041 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/ae9d0d08-59de-454d-b7c5-45fc0393a7f3-node-exporter-tls") pod "node-exporter-2k2gh" (UID: "ae9d0d08-59de-454d-b7c5-45fc0393a7f3") : secret "node-exporter-tls" not found
Sep 17 16:43:10.393440 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.393133    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ae9d0d08-59de-454d-b7c5-45fc0393a7f3-node-exporter-textfile\") pod \"node-exporter-2k2gh\" (UID: \"ae9d0d08-59de-454d-b7c5-45fc0393a7f3\") " pod="openshift-monitoring/node-exporter-2k2gh"
Sep 17 16:43:10.393440 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.393179    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ae9d0d08-59de-454d-b7c5-45fc0393a7f3-node-exporter-wtmp\") pod \"node-exporter-2k2gh\" (UID: \"ae9d0d08-59de-454d-b7c5-45fc0393a7f3\") " pod="openshift-monitoring/node-exporter-2k2gh"
Sep 17 16:43:10.393440 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.393412    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ae9d0d08-59de-454d-b7c5-45fc0393a7f3-node-exporter-accelerators-collector-config\") pod \"node-exporter-2k2gh\" (UID: \"ae9d0d08-59de-454d-b7c5-45fc0393a7f3\") " pod="openshift-monitoring/node-exporter-2k2gh"
Sep 17 16:43:10.393591 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.393524    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae9d0d08-59de-454d-b7c5-45fc0393a7f3-metrics-client-ca\") pod \"node-exporter-2k2gh\" (UID: \"ae9d0d08-59de-454d-b7c5-45fc0393a7f3\") " pod="openshift-monitoring/node-exporter-2k2gh"
Sep 17 16:43:10.395545 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.395520    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae9d0d08-59de-454d-b7c5-45fc0393a7f3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2k2gh\" (UID: \"ae9d0d08-59de-454d-b7c5-45fc0393a7f3\") " pod="openshift-monitoring/node-exporter-2k2gh"
Sep 17 16:43:10.412588 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.412560    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv9bs\" (UniqueName: \"kubernetes.io/projected/ae9d0d08-59de-454d-b7c5-45fc0393a7f3-kube-api-access-fv9bs\") pod \"node-exporter-2k2gh\" (UID: \"ae9d0d08-59de-454d-b7c5-45fc0393a7f3\") " pod="openshift-monitoring/node-exporter-2k2gh"
Sep 17 16:43:10.472143 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.472112    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-22fdn"
Sep 17 16:43:10.897740 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.897699    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ae9d0d08-59de-454d-b7c5-45fc0393a7f3-node-exporter-tls\") pod \"node-exporter-2k2gh\" (UID: \"ae9d0d08-59de-454d-b7c5-45fc0393a7f3\") " pod="openshift-monitoring/node-exporter-2k2gh"
Sep 17 16:43:10.900268 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:10.900242    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ae9d0d08-59de-454d-b7c5-45fc0393a7f3-node-exporter-tls\") pod \"node-exporter-2k2gh\" (UID: \"ae9d0d08-59de-454d-b7c5-45fc0393a7f3\") " pod="openshift-monitoring/node-exporter-2k2gh"
Sep 17 16:43:11.185707 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.185617    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2k2gh"
Sep 17 16:43:11.195190 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:43:11.195157    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae9d0d08_59de_454d_b7c5_45fc0393a7f3.slice/crio-1a178eccccedcaf364102043f799d10e638deaad85c93db9246515cb3d4aa3b0 WatchSource:0}: Error finding container 1a178eccccedcaf364102043f799d10e638deaad85c93db9246515cb3d4aa3b0: Status 404 returned error can't find the container with id 1a178eccccedcaf364102043f799d10e638deaad85c93db9246515cb3d4aa3b0
Sep 17 16:43:11.360530 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.360492    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 16:43:11.365123 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.365097    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.367894 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.367845    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 17 16:43:11.368094 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.367927    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 17 16:43:11.368094 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.367931    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 17 16:43:11.368094 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.367957    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 17 16:43:11.368280 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.368219    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 17 16:43:11.368280 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.368225    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 17 16:43:11.368411 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.368336    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 17 16:43:11.368411 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.368371    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 17 16:43:11.368551 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.368432    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 17 16:43:11.368616 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.368565    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-wprm8\""
Sep 17 16:43:11.378611 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.378591    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 16:43:11.401436 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.401407    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-web-config\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.401615 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.401455    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.401615 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.401526    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a772efd-a0f1-4d09-afe7-72805b37ab2e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.401615 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.401576    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4a772efd-a0f1-4d09-afe7-72805b37ab2e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.401615 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.401605    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a772efd-a0f1-4d09-afe7-72805b37ab2e-config-out\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.401831 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.401629    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a772efd-a0f1-4d09-afe7-72805b37ab2e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.401831 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.401701    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a772efd-a0f1-4d09-afe7-72805b37ab2e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.401831 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.401737    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.401831 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.401774    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.401831 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.401820    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-config-volume\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.402311 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.401849    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.402311 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.401903    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.402311 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.402124    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sr69\" (UniqueName: \"kubernetes.io/projected/4a772efd-a0f1-4d09-afe7-72805b37ab2e-kube-api-access-2sr69\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.502946 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.502830    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-web-config\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.502946 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.502910    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.503179 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.502950    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a772efd-a0f1-4d09-afe7-72805b37ab2e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.503179 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.502978    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4a772efd-a0f1-4d09-afe7-72805b37ab2e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.503179 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.503005    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a772efd-a0f1-4d09-afe7-72805b37ab2e-config-out\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.503179 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:43:11.503054    2586 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Sep 17 16:43:11.503179 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.503118    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a772efd-a0f1-4d09-afe7-72805b37ab2e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.503179 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:43:11.503129    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-secret-alertmanager-main-tls podName:4a772efd-a0f1-4d09-afe7-72805b37ab2e nodeName:}" failed. No retries permitted until 2026-09-17 16:43:12.003107155 +0000 UTC m=+62.704185817 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "4a772efd-a0f1-4d09-afe7-72805b37ab2e") : secret "alertmanager-main-tls" not found
Sep 17 16:43:11.503179 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.503170    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a772efd-a0f1-4d09-afe7-72805b37ab2e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.503526 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:43:11.503283    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4a772efd-a0f1-4d09-afe7-72805b37ab2e-alertmanager-trusted-ca-bundle podName:4a772efd-a0f1-4d09-afe7-72805b37ab2e nodeName:}" failed. No retries permitted until 2026-09-17 16:43:12.003267652 +0000 UTC m=+62.704346308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/4a772efd-a0f1-4d09-afe7-72805b37ab2e-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "4a772efd-a0f1-4d09-afe7-72805b37ab2e") : configmap references non-existent config key: ca-bundle.crt
Sep 17 16:43:11.503526 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.503306    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.503526 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.503333    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.503526 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.503379    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-config-volume\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.503526 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.503389    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4a772efd-a0f1-4d09-afe7-72805b37ab2e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.503526 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.503409    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.503526 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.503452    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.503526 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.503482    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sr69\" (UniqueName: \"kubernetes.io/projected/4a772efd-a0f1-4d09-afe7-72805b37ab2e-kube-api-access-2sr69\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.504116 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.503956    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a772efd-a0f1-4d09-afe7-72805b37ab2e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.506893 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.506655    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a772efd-a0f1-4d09-afe7-72805b37ab2e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.506893 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.506802    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.506893 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.506805    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.507081 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.506899    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.507081 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.506954    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-config-volume\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.507081 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.507023    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.507169 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.507147    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-web-config\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.507213 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.507197    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a772efd-a0f1-4d09-afe7-72805b37ab2e-config-out\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.511812 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.511788    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sr69\" (UniqueName: \"kubernetes.io/projected/4a772efd-a0f1-4d09-afe7-72805b37ab2e-kube-api-access-2sr69\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:11.603943 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:11.603907    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2k2gh" event={"ID":"ae9d0d08-59de-454d-b7c5-45fc0393a7f3","Type":"ContainerStarted","Data":"1a178eccccedcaf364102043f799d10e638deaad85c93db9246515cb3d4aa3b0"}
Sep 17 16:43:12.007950 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:12.007912    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:12.008106 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:12.007964    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a772efd-a0f1-4d09-afe7-72805b37ab2e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:12.008736 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:12.008717    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a772efd-a0f1-4d09-afe7-72805b37ab2e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:12.010599 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:12.010576    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:12.268193 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:12.268045    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6db46dc76b-cnrnc"
Sep 17 16:43:12.268193 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:12.268175    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6db46dc76b-cnrnc"
Sep 17 16:43:12.274938 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:12.274908    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6db46dc76b-cnrnc"
Sep 17 16:43:12.275138 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:12.275059    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:43:12.565604 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:12.565571    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 16:43:12.611602 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:12.611570    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6db46dc76b-cnrnc"
Sep 17 16:43:12.678275 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:43:12.678230    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a772efd_a0f1_4d09_afe7_72805b37ab2e.slice/crio-ca0d18732794b587cc82a5a18f2ab91308d94d5897977e769d2b8b541043dc6c WatchSource:0}: Error finding container ca0d18732794b587cc82a5a18f2ab91308d94d5897977e769d2b8b541043dc6c: Status 404 returned error can't find the container with id ca0d18732794b587cc82a5a18f2ab91308d94d5897977e769d2b8b541043dc6c
Sep 17 16:43:13.495368 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:13.495330    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76795db869-sm9jc"]
Sep 17 16:43:13.497597 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:13.497568    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76795db869-sm9jc"
Sep 17 16:43:13.507778 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:13.507751    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 17 16:43:13.510274 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:13.510216    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76795db869-sm9jc"]
Sep 17 16:43:13.523766 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:13.523741    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-console-oauth-config\") pod \"console-76795db869-sm9jc\" (UID: \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\") " pod="openshift-console/console-76795db869-sm9jc"
Sep 17 16:43:13.523934 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:13.523801    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-oauth-serving-cert\") pod \"console-76795db869-sm9jc\" (UID: \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\") " pod="openshift-console/console-76795db869-sm9jc"
Sep 17 16:43:13.523934 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:13.523830    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj5xr\" (UniqueName: \"kubernetes.io/projected/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-kube-api-access-kj5xr\") pod \"console-76795db869-sm9jc\" (UID: \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\") " pod="openshift-console/console-76795db869-sm9jc"
Sep 17 16:43:13.524064 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:13.523930    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-service-ca\") pod \"console-76795db869-sm9jc\" (UID: \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\") " pod="openshift-console/console-76795db869-sm9jc"
Sep 17 16:43:13.524064 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:13.523975    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-console-serving-cert\") pod \"console-76795db869-sm9jc\" (UID: \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\") " pod="openshift-console/console-76795db869-sm9jc"
Sep 17 16:43:13.524064 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:13.523991    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-console-config\") pod \"console-76795db869-sm9jc\" (UID: \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\") " pod="openshift-console/console-76795db869-sm9jc"
Sep 17 16:43:13.524215 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:13.524074    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-trusted-ca-bundle\") pod \"console-76795db869-sm9jc\" (UID: \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\") " pod="openshift-console/console-76795db869-sm9jc"
Sep 17 16:43:13.612178 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:13.612137    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a772efd-a0f1-4d09-afe7-72805b37ab2e","Type":"ContainerStarted","Data":"ca0d18732794b587cc82a5a18f2ab91308d94d5897977e769d2b8b541043dc6c"}
Sep 17 16:43:13.614147 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:13.614112    2586 generic.go:356] "Generic (PLEG): container finished" podID="ae9d0d08-59de-454d-b7c5-45fc0393a7f3" containerID="b3aa13ce354d4bc60b5074d108bcd5320ab733228225de965e52c6a20acc17f5" exitCode=0
Sep 17 16:43:13.614274 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:13.614193    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2k2gh" event={"ID":"ae9d0d08-59de-454d-b7c5-45fc0393a7f3","Type":"ContainerDied","Data":"b3aa13ce354d4bc60b5074d108bcd5320ab733228225de965e52c6a20acc17f5"}
Sep 17 16:43:13.625015 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:13.624987    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-oauth-serving-cert\") pod \"console-76795db869-sm9jc\" (UID: \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\") " pod="openshift-console/console-76795db869-sm9jc"
Sep 17 16:43:13.625140 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:13.625027    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kj5xr\" (UniqueName: \"kubernetes.io/projected/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-kube-api-access-kj5xr\") pod \"console-76795db869-sm9jc\" (UID: \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\") " pod="openshift-console/console-76795db869-sm9jc"
Sep 17 16:43:13.625140 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:13.625063    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-service-ca\") pod \"console-76795db869-sm9jc\" (UID: \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\") " pod="openshift-console/console-76795db869-sm9jc"
Sep 17 16:43:13.625140 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:13.625096    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-console-serving-cert\") pod \"console-76795db869-sm9jc\" (UID: \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\") " pod="openshift-console/console-76795db869-sm9jc"
Sep 17 16:43:13.625140 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:13.625120    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-console-config\") pod \"console-76795db869-sm9jc\" (UID: \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\") " pod="openshift-console/console-76795db869-sm9jc"
Sep 17 16:43:13.625353 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:13.625328    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-trusted-ca-bundle\") pod \"console-76795db869-sm9jc\" (UID: \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\") " pod="openshift-console/console-76795db869-sm9jc"
Sep 17 16:43:13.625424 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:13.625405    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-console-oauth-config\") pod \"console-76795db869-sm9jc\" (UID: \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\") " pod="openshift-console/console-76795db869-sm9jc"
Sep 17 16:43:13.625810 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:13.625780    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-oauth-serving-cert\") pod \"console-76795db869-sm9jc\" (UID: \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\") " pod="openshift-console/console-76795db869-sm9jc"
Sep 17 16:43:13.625916 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:13.625852    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-service-ca\") pod \"console-76795db869-sm9jc\" (UID: \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\") " pod="openshift-console/console-76795db869-sm9jc"
Sep 17 16:43:13.625916 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:13.625902    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-console-config\") pod \"console-76795db869-sm9jc\" (UID: \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\") " pod="openshift-console/console-76795db869-sm9jc"
Sep 17 16:43:13.626326 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:13.626303    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-trusted-ca-bundle\") pod \"console-76795db869-sm9jc\" (UID: \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\") " pod="openshift-console/console-76795db869-sm9jc"
Sep 17 16:43:13.628087 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:13.628063    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-console-oauth-config\") pod \"console-76795db869-sm9jc\" (UID: \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\") " pod="openshift-console/console-76795db869-sm9jc"
Sep 17 16:43:13.628168 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:13.628106    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-console-serving-cert\") pod \"console-76795db869-sm9jc\" (UID: \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\") " pod="openshift-console/console-76795db869-sm9jc"
Sep 17 16:43:13.644396 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:13.644361    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj5xr\" (UniqueName: \"kubernetes.io/projected/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-kube-api-access-kj5xr\") pod \"console-76795db869-sm9jc\" (UID: \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\") " pod="openshift-console/console-76795db869-sm9jc"
Sep 17 16:43:13.810152 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:13.810106    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76795db869-sm9jc"
Sep 17 16:43:14.100043 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:14.099955    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76795db869-sm9jc"]
Sep 17 16:43:14.102599 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:43:14.102574    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a688c1b_d943_4c51_b42f_b7c727b7ffc2.slice/crio-5fdfe94f890027b746458c6d36307561ca1cfb6d0824c36085f664ecc42920fa WatchSource:0}: Error finding container 5fdfe94f890027b746458c6d36307561ca1cfb6d0824c36085f664ecc42920fa: Status 404 returned error can't find the container with id 5fdfe94f890027b746458c6d36307561ca1cfb6d0824c36085f664ecc42920fa
Sep 17 16:43:14.578809 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:14.578782    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pnvqx"
Sep 17 16:43:14.630993 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:14.630955    2586 generic.go:356] "Generic (PLEG): container finished" podID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerID="af0b4f3ac48be7f3d0482db5b50658413b00e5000801a7d93e41f85a3379c639" exitCode=0
Sep 17 16:43:14.631459 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:14.631064    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a772efd-a0f1-4d09-afe7-72805b37ab2e","Type":"ContainerDied","Data":"af0b4f3ac48be7f3d0482db5b50658413b00e5000801a7d93e41f85a3379c639"}
Sep 17 16:43:14.639719 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:14.639690    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2k2gh" event={"ID":"ae9d0d08-59de-454d-b7c5-45fc0393a7f3","Type":"ContainerStarted","Data":"d703a9afd96d4bf5f222a6bd6d1a55560320f0dd66c78ad6562b84f7fcefacd2"}
Sep 17 16:43:14.639805 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:14.639787    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2k2gh" event={"ID":"ae9d0d08-59de-454d-b7c5-45fc0393a7f3","Type":"ContainerStarted","Data":"49f6bd1c2c43f057f61f96db2dfefe6772279b5dce829a678911607161878742"}
Sep 17 16:43:14.641993 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:14.641967    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76795db869-sm9jc" event={"ID":"0a688c1b-d943-4c51-b42f-b7c727b7ffc2","Type":"ContainerStarted","Data":"df8abb6ec2b86aad91bc89a6a856d044745b8ec23f6df7221079254e4b80db68"}
Sep 17 16:43:14.642101 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:14.642014    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76795db869-sm9jc" event={"ID":"0a688c1b-d943-4c51-b42f-b7c727b7ffc2","Type":"ContainerStarted","Data":"5fdfe94f890027b746458c6d36307561ca1cfb6d0824c36085f664ecc42920fa"}
Sep 17 16:43:14.680295 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:14.680247    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2k2gh" podStartSLOduration=3.103615782 podStartE2EDuration="4.680231253s" totalImagesPullingTime="1.504290967s" totalInitContainerRuntime="72.324504ms" isStatefulPod=true podCreationTimestamp="2026-09-17 16:43:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:43:14.679191955 +0000 UTC m=+65.380270617" watchObservedRunningTime="2026-09-17 16:43:14.680231253 +0000 UTC m=+65.381309955"
Sep 17 16:43:14.696838 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:14.696784    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76795db869-sm9jc" podStartSLOduration=1.696773171 podStartE2EDuration="1.696773171s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:43:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:43:14.695583316 +0000 UTC m=+65.396661980" watchObservedRunningTime="2026-09-17 16:43:14.696773171 +0000 UTC m=+65.397851833"
Sep 17 16:43:16.656693 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:16.656652    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a772efd-a0f1-4d09-afe7-72805b37ab2e","Type":"ContainerStarted","Data":"ac59d045104ca38f10db38ed02966cd3b1d70945b4db47b4bbd71c53ad6a80b9"}
Sep 17 16:43:16.657143 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:16.656742    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a772efd-a0f1-4d09-afe7-72805b37ab2e","Type":"ContainerStarted","Data":"912ddc4deea35fef0a9f63c2375ecda6d3a1e0384afffabe7456fd5192de3b60"}
Sep 17 16:43:16.657143 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:16.656783    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a772efd-a0f1-4d09-afe7-72805b37ab2e","Type":"ContainerStarted","Data":"65f556a0813ac886b5b9a4b3ee93aa42b881a7fa428146d0d4afa8136f234f64"}
Sep 17 16:43:16.657143 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:16.656835    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a772efd-a0f1-4d09-afe7-72805b37ab2e","Type":"ContainerStarted","Data":"9b65810bb5a87b5061386f85df47e0eca44d7e3d4796337968aca73a0b9059b1"}
Sep 17 16:43:16.657143 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:16.656895    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a772efd-a0f1-4d09-afe7-72805b37ab2e","Type":"ContainerStarted","Data":"2aaf886d418c93e4e4a047aff33a1bea5ab67054ad0f4debd83a4bf8360ee06d"}
Sep 17 16:43:17.057149 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:17.057103    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3d4b959-b664-439d-b7a8-f4120b31a45d-metrics-certs\") pod \"network-metrics-daemon-z77cp\" (UID: \"f3d4b959-b664-439d-b7a8-f4120b31a45d\") " pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:43:17.059841 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:17.059817    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 16:43:17.070389 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:17.070356    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3d4b959-b664-439d-b7a8-f4120b31a45d-metrics-certs\") pod \"network-metrics-daemon-z77cp\" (UID: \"f3d4b959-b664-439d-b7a8-f4120b31a45d\") " pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:43:17.157801 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:17.157762    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/36250538-b6ed-4b8d-8788-a205a4dd53c6-original-pull-secret\") pod \"global-pull-secret-syncer-jpzxf\" (UID: \"36250538-b6ed-4b8d-8788-a205a4dd53c6\") " pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:43:17.157984 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:17.157811    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9t5z\" (UniqueName: \"kubernetes.io/projected/c3e3bba0-3038-411b-8194-5a88b7a0f443-kube-api-access-k9t5z\") pod \"network-check-target-nxc7t\" (UID: \"c3e3bba0-3038-411b-8194-5a88b7a0f443\") " pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:43:17.160499 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:17.160478    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 16:43:17.160554 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:17.160479    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 16:43:17.171115 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:17.171082    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 16:43:17.171263 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:17.171245    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/36250538-b6ed-4b8d-8788-a205a4dd53c6-original-pull-secret\") pod \"global-pull-secret-syncer-jpzxf\" (UID: \"36250538-b6ed-4b8d-8788-a205a4dd53c6\") " pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:43:17.182130 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:17.182098    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9t5z\" (UniqueName: \"kubernetes.io/projected/c3e3bba0-3038-411b-8194-5a88b7a0f443-kube-api-access-k9t5z\") pod \"network-check-target-nxc7t\" (UID: \"c3e3bba0-3038-411b-8194-5a88b7a0f443\") " pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:43:17.238066 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:17.238037    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kh76t\""
Sep 17 16:43:17.245125 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:17.245105    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:43:17.245195 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:17.245139    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ls2kp\""
Sep 17 16:43:17.248585 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:17.248565    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jpzxf"
Sep 17 16:43:17.253736 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:17.253716    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z77cp"
Sep 17 16:43:17.538194 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:17.538159    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nxc7t"]
Sep 17 16:43:17.541192 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:43:17.541164    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3e3bba0_3038_411b_8194_5a88b7a0f443.slice/crio-72fae0f3785736669b5aae96f9d4a7bd514a1888dee8db2111062c8250087753 WatchSource:0}: Error finding container 72fae0f3785736669b5aae96f9d4a7bd514a1888dee8db2111062c8250087753: Status 404 returned error can't find the container with id 72fae0f3785736669b5aae96f9d4a7bd514a1888dee8db2111062c8250087753
Sep 17 16:43:17.574755 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:17.574691    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jpzxf"]
Sep 17 16:43:17.578993 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:43:17.578956    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36250538_b6ed_4b8d_8788_a205a4dd53c6.slice/crio-e58f9fa3e4b16743651d112b0af9c56be16a85bcad9453e1104e6288d4df3814 WatchSource:0}: Error finding container e58f9fa3e4b16743651d112b0af9c56be16a85bcad9453e1104e6288d4df3814: Status 404 returned error can't find the container with id e58f9fa3e4b16743651d112b0af9c56be16a85bcad9453e1104e6288d4df3814
Sep 17 16:43:17.591700 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:17.591676    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-z77cp"]
Sep 17 16:43:17.594712 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:43:17.594690    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3d4b959_b664_439d_b7a8_f4120b31a45d.slice/crio-9c74347acf8206b798b6bcbb77e468a1347610fc462026f7a9f45b909e3cef2a WatchSource:0}: Error finding container 9c74347acf8206b798b6bcbb77e468a1347610fc462026f7a9f45b909e3cef2a: Status 404 returned error can't find the container with id 9c74347acf8206b798b6bcbb77e468a1347610fc462026f7a9f45b909e3cef2a
Sep 17 16:43:17.663837 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:17.663798    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a772efd-a0f1-4d09-afe7-72805b37ab2e","Type":"ContainerStarted","Data":"7fc8f3221de365777f83751714ce1c51d308115f62d8adaaf0a1504bca430850"}
Sep 17 16:43:17.665116 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:17.665086    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-z77cp" event={"ID":"f3d4b959-b664-439d-b7a8-f4120b31a45d","Type":"ContainerStarted","Data":"9c74347acf8206b798b6bcbb77e468a1347610fc462026f7a9f45b909e3cef2a"}
Sep 17 16:43:17.666458 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:17.666436    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nxc7t" event={"ID":"c3e3bba0-3038-411b-8194-5a88b7a0f443","Type":"ContainerStarted","Data":"72fae0f3785736669b5aae96f9d4a7bd514a1888dee8db2111062c8250087753"}
Sep 17 16:43:17.667599 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:17.667579    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jpzxf" event={"ID":"36250538-b6ed-4b8d-8788-a205a4dd53c6","Type":"ContainerStarted","Data":"e58f9fa3e4b16743651d112b0af9c56be16a85bcad9453e1104e6288d4df3814"}
Sep 17 16:43:17.693496 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:17.693437    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.222587941 podStartE2EDuration="6.693415342s" totalImagesPullingTime="3.445101107s" totalInitContainerRuntime="25.726294ms" isStatefulPod=false podCreationTimestamp="2026-09-17 16:43:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:43:17.691358048 +0000 UTC m=+68.392436710" watchObservedRunningTime="2026-09-17 16:43:17.693415342 +0000 UTC m=+68.394494005"
Sep 17 16:43:19.679917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:19.679121    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-z77cp" event={"ID":"f3d4b959-b664-439d-b7a8-f4120b31a45d","Type":"ContainerStarted","Data":"2a950e1cc814d7d41a6570864b22c53eb22395147321d55cf9bee90968ee25d3"}
Sep 17 16:43:19.679917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:19.679194    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-z77cp" event={"ID":"f3d4b959-b664-439d-b7a8-f4120b31a45d","Type":"ContainerStarted","Data":"d340c2890696e5031f0a9a49f9e8cb4faf1315170d2e9adec39a9fdf2f68d12d"}
Sep 17 16:43:19.701150 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:19.701087    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-z77cp" podStartSLOduration=67.114787526 podStartE2EDuration="1m8.701064259s" totalImagesPullingTime="1.586276733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:42:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:43:19.699829993 +0000 UTC m=+70.400908655" watchObservedRunningTime="2026-09-17 16:43:19.701064259 +0000 UTC m=+70.402142922"
Sep 17 16:43:22.536519 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:22.536385    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6b849696d6-2lzx6"
Sep 17 16:43:23.110646 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.110605    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76795db869-sm9jc"]
Sep 17 16:43:23.143421 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.143386    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f87766945-8gkhv"]
Sep 17 16:43:23.146415 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.146397    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f87766945-8gkhv"
Sep 17 16:43:23.157938 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.157908    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f87766945-8gkhv"]
Sep 17 16:43:23.212977 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.212939    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/04213207-3bca-4a89-882a-1cda4ae924f1-console-serving-cert\") pod \"console-7f87766945-8gkhv\" (UID: \"04213207-3bca-4a89-882a-1cda4ae924f1\") " pod="openshift-console/console-7f87766945-8gkhv"
Sep 17 16:43:23.213138 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.212989    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/04213207-3bca-4a89-882a-1cda4ae924f1-oauth-serving-cert\") pod \"console-7f87766945-8gkhv\" (UID: \"04213207-3bca-4a89-882a-1cda4ae924f1\") " pod="openshift-console/console-7f87766945-8gkhv"
Sep 17 16:43:23.213138 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.213047    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04213207-3bca-4a89-882a-1cda4ae924f1-trusted-ca-bundle\") pod \"console-7f87766945-8gkhv\" (UID: \"04213207-3bca-4a89-882a-1cda4ae924f1\") " pod="openshift-console/console-7f87766945-8gkhv"
Sep 17 16:43:23.213138 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.213075    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/04213207-3bca-4a89-882a-1cda4ae924f1-console-oauth-config\") pod \"console-7f87766945-8gkhv\" (UID: \"04213207-3bca-4a89-882a-1cda4ae924f1\") " pod="openshift-console/console-7f87766945-8gkhv"
Sep 17 16:43:23.213138 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.213097    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/04213207-3bca-4a89-882a-1cda4ae924f1-console-config\") pod \"console-7f87766945-8gkhv\" (UID: \"04213207-3bca-4a89-882a-1cda4ae924f1\") " pod="openshift-console/console-7f87766945-8gkhv"
Sep 17 16:43:23.213138 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.213113    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsq54\" (UniqueName: \"kubernetes.io/projected/04213207-3bca-4a89-882a-1cda4ae924f1-kube-api-access-rsq54\") pod \"console-7f87766945-8gkhv\" (UID: \"04213207-3bca-4a89-882a-1cda4ae924f1\") " pod="openshift-console/console-7f87766945-8gkhv"
Sep 17 16:43:23.213138 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.213127    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/04213207-3bca-4a89-882a-1cda4ae924f1-service-ca\") pod \"console-7f87766945-8gkhv\" (UID: \"04213207-3bca-4a89-882a-1cda4ae924f1\") " pod="openshift-console/console-7f87766945-8gkhv"
Sep 17 16:43:23.314355 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.314316    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/04213207-3bca-4a89-882a-1cda4ae924f1-console-serving-cert\") pod \"console-7f87766945-8gkhv\" (UID: \"04213207-3bca-4a89-882a-1cda4ae924f1\") " pod="openshift-console/console-7f87766945-8gkhv"
Sep 17 16:43:23.314561 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.314367    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/04213207-3bca-4a89-882a-1cda4ae924f1-oauth-serving-cert\") pod \"console-7f87766945-8gkhv\" (UID: \"04213207-3bca-4a89-882a-1cda4ae924f1\") " pod="openshift-console/console-7f87766945-8gkhv"
Sep 17 16:43:23.314561 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.314394    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04213207-3bca-4a89-882a-1cda4ae924f1-trusted-ca-bundle\") pod \"console-7f87766945-8gkhv\" (UID: \"04213207-3bca-4a89-882a-1cda4ae924f1\") " pod="openshift-console/console-7f87766945-8gkhv"
Sep 17 16:43:23.314561 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.314412    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/04213207-3bca-4a89-882a-1cda4ae924f1-console-oauth-config\") pod \"console-7f87766945-8gkhv\" (UID: \"04213207-3bca-4a89-882a-1cda4ae924f1\") " pod="openshift-console/console-7f87766945-8gkhv"
Sep 17 16:43:23.314561 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.314427    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/04213207-3bca-4a89-882a-1cda4ae924f1-console-config\") pod \"console-7f87766945-8gkhv\" (UID: \"04213207-3bca-4a89-882a-1cda4ae924f1\") " pod="openshift-console/console-7f87766945-8gkhv"
Sep 17 16:43:23.314561 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.314443    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsq54\" (UniqueName: \"kubernetes.io/projected/04213207-3bca-4a89-882a-1cda4ae924f1-kube-api-access-rsq54\") pod \"console-7f87766945-8gkhv\" (UID: \"04213207-3bca-4a89-882a-1cda4ae924f1\") " pod="openshift-console/console-7f87766945-8gkhv"
Sep 17 16:43:23.314561 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.314458    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/04213207-3bca-4a89-882a-1cda4ae924f1-service-ca\") pod \"console-7f87766945-8gkhv\" (UID: \"04213207-3bca-4a89-882a-1cda4ae924f1\") " pod="openshift-console/console-7f87766945-8gkhv"
Sep 17 16:43:23.315321 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.315225    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/04213207-3bca-4a89-882a-1cda4ae924f1-oauth-serving-cert\") pod \"console-7f87766945-8gkhv\" (UID: \"04213207-3bca-4a89-882a-1cda4ae924f1\") " pod="openshift-console/console-7f87766945-8gkhv"
Sep 17 16:43:23.315500 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.315389    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04213207-3bca-4a89-882a-1cda4ae924f1-trusted-ca-bundle\") pod \"console-7f87766945-8gkhv\" (UID: \"04213207-3bca-4a89-882a-1cda4ae924f1\") " pod="openshift-console/console-7f87766945-8gkhv"
Sep 17 16:43:23.315633 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.315608    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/04213207-3bca-4a89-882a-1cda4ae924f1-console-config\") pod \"console-7f87766945-8gkhv\" (UID: \"04213207-3bca-4a89-882a-1cda4ae924f1\") " pod="openshift-console/console-7f87766945-8gkhv"
Sep 17 16:43:23.315901 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.315837    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/04213207-3bca-4a89-882a-1cda4ae924f1-service-ca\") pod \"console-7f87766945-8gkhv\" (UID: \"04213207-3bca-4a89-882a-1cda4ae924f1\") " pod="openshift-console/console-7f87766945-8gkhv"
Sep 17 16:43:23.318031 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.317516    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/04213207-3bca-4a89-882a-1cda4ae924f1-console-serving-cert\") pod \"console-7f87766945-8gkhv\" (UID: \"04213207-3bca-4a89-882a-1cda4ae924f1\") " pod="openshift-console/console-7f87766945-8gkhv"
Sep 17 16:43:23.318031 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.317519    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/04213207-3bca-4a89-882a-1cda4ae924f1-console-oauth-config\") pod \"console-7f87766945-8gkhv\" (UID: \"04213207-3bca-4a89-882a-1cda4ae924f1\") " pod="openshift-console/console-7f87766945-8gkhv"
Sep 17 16:43:23.329350 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.329321    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsq54\" (UniqueName: \"kubernetes.io/projected/04213207-3bca-4a89-882a-1cda4ae924f1-kube-api-access-rsq54\") pod \"console-7f87766945-8gkhv\" (UID: \"04213207-3bca-4a89-882a-1cda4ae924f1\") " pod="openshift-console/console-7f87766945-8gkhv"
Sep 17 16:43:23.455542 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.455440    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f87766945-8gkhv"
Sep 17 16:43:23.699962 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.699911    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nxc7t" event={"ID":"c3e3bba0-3038-411b-8194-5a88b7a0f443","Type":"ContainerStarted","Data":"e3974f61bd69f72eed41970afb2195c56e5386ebfe946bbd3426fa9281a49aa5"}
Sep 17 16:43:23.700419 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.699988    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:43:23.701737 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.701703    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jpzxf" event={"ID":"36250538-b6ed-4b8d-8788-a205a4dd53c6","Type":"ContainerStarted","Data":"2b48bdaf20aa7a098bce2c607fbbe84a5d39ff8e47dd9e07b27c0340068e87a5"}
Sep 17 16:43:23.717578 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.717466    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-nxc7t" podStartSLOduration=67.624460327 podStartE2EDuration="1m12.717445588s" totalImagesPullingTime="5.092985261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:42:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:43:23.715855026 +0000 UTC m=+74.416933689" watchObservedRunningTime="2026-09-17 16:43:23.717445588 +0000 UTC m=+74.418524266"
Sep 17 16:43:23.734473 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.734422    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-jpzxf" podStartSLOduration=67.679474631 podStartE2EDuration="1m12.734405656s" totalImagesPullingTime="5.054931025s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 16:42:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:43:23.732781441 +0000 UTC m=+74.433860104" watchObservedRunningTime="2026-09-17 16:43:23.734405656 +0000 UTC m=+74.435484318"
Sep 17 16:43:23.737975 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.737946    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f87766945-8gkhv"]
Sep 17 16:43:23.741731 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:43:23.741698    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04213207_3bca_4a89_882a_1cda4ae924f1.slice/crio-7f6e38ff9e5ea3e43054e3dae4e2d3f837177900f90e0be747bc37ac6155d9d8 WatchSource:0}: Error finding container 7f6e38ff9e5ea3e43054e3dae4e2d3f837177900f90e0be747bc37ac6155d9d8: Status 404 returned error can't find the container with id 7f6e38ff9e5ea3e43054e3dae4e2d3f837177900f90e0be747bc37ac6155d9d8
Sep 17 16:43:23.810290 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:23.810268    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76795db869-sm9jc"
Sep 17 16:43:24.708050 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:24.708001    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f87766945-8gkhv" event={"ID":"04213207-3bca-4a89-882a-1cda4ae924f1","Type":"ContainerStarted","Data":"7765a4d104608b6ade605e8d25879154cc41687bcb90e5e5cae287a9fc97d0d2"}
Sep 17 16:43:24.708523 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:24.708065    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f87766945-8gkhv" event={"ID":"04213207-3bca-4a89-882a-1cda4ae924f1","Type":"ContainerStarted","Data":"7f6e38ff9e5ea3e43054e3dae4e2d3f837177900f90e0be747bc37ac6155d9d8"}
Sep 17 16:43:24.726120 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:24.726070    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f87766945-8gkhv" podStartSLOduration=1.726052347 podStartE2EDuration="1.726052347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:43:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:43:24.725352229 +0000 UTC m=+75.426430890" watchObservedRunningTime="2026-09-17 16:43:24.726052347 +0000 UTC m=+75.427131009"
Sep 17 16:43:33.456400 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:33.456358    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f87766945-8gkhv"
Sep 17 16:43:33.456911 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:33.456431    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f87766945-8gkhv"
Sep 17 16:43:33.461195 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:33.461173    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f87766945-8gkhv"
Sep 17 16:43:33.746065 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:33.745979    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f87766945-8gkhv"
Sep 17 16:43:33.796585 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:33.796543    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6db46dc76b-cnrnc"]
Sep 17 16:43:48.130152 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.130085    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-76795db869-sm9jc" podUID="0a688c1b-d943-4c51-b42f-b7c727b7ffc2" containerName="console" containerID="cri-o://df8abb6ec2b86aad91bc89a6a856d044745b8ec23f6df7221079254e4b80db68" gracePeriod=15
Sep 17 16:43:48.368827 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.368800    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76795db869-sm9jc"
Sep 17 16:43:48.528783 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.528751    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-service-ca\") pod \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\" (UID: \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\") "
Sep 17 16:43:48.528985 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.528791    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-console-config\") pod \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\" (UID: \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\") "
Sep 17 16:43:48.528985 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.528833    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-console-serving-cert\") pod \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\" (UID: \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\") "
Sep 17 16:43:48.528985 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.528859    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-oauth-serving-cert\") pod \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\" (UID: \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\") "
Sep 17 16:43:48.528985 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.528956    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-console-oauth-config\") pod \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\" (UID: \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\") "
Sep 17 16:43:48.528985 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.528973    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-trusted-ca-bundle\") pod \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\" (UID: \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\") "
Sep 17 16:43:48.529248 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.529006    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kj5xr\" (UniqueName: \"kubernetes.io/projected/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-kube-api-access-kj5xr\") pod \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\" (UID: \"0a688c1b-d943-4c51-b42f-b7c727b7ffc2\") "
Sep 17 16:43:48.529402 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.529372    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-console-config" (OuterVolumeSpecName: "console-config") pod "0a688c1b-d943-4c51-b42f-b7c727b7ffc2" (UID: "0a688c1b-d943-4c51-b42f-b7c727b7ffc2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:43:48.529402 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.529376    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-service-ca" (OuterVolumeSpecName: "service-ca") pod "0a688c1b-d943-4c51-b42f-b7c727b7ffc2" (UID: "0a688c1b-d943-4c51-b42f-b7c727b7ffc2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:43:48.529591 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.529476    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0a688c1b-d943-4c51-b42f-b7c727b7ffc2" (UID: "0a688c1b-d943-4c51-b42f-b7c727b7ffc2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:43:48.529591 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.529495    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0a688c1b-d943-4c51-b42f-b7c727b7ffc2" (UID: "0a688c1b-d943-4c51-b42f-b7c727b7ffc2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:43:48.531478 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.531453    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-kube-api-access-kj5xr" (OuterVolumeSpecName: "kube-api-access-kj5xr") pod "0a688c1b-d943-4c51-b42f-b7c727b7ffc2" (UID: "0a688c1b-d943-4c51-b42f-b7c727b7ffc2"). InnerVolumeSpecName "kube-api-access-kj5xr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:43:48.531574 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.531468    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0a688c1b-d943-4c51-b42f-b7c727b7ffc2" (UID: "0a688c1b-d943-4c51-b42f-b7c727b7ffc2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:43:48.531574 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.531522    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0a688c1b-d943-4c51-b42f-b7c727b7ffc2" (UID: "0a688c1b-d943-4c51-b42f-b7c727b7ffc2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:43:48.630025 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.629988    2586 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-console-serving-cert\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:43:48.630025 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.630019    2586 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-oauth-serving-cert\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:43:48.630025 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.630028    2586 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-console-oauth-config\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:43:48.630025 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.630036    2586 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-trusted-ca-bundle\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:43:48.630295 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.630045    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kj5xr\" (UniqueName: \"kubernetes.io/projected/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-kube-api-access-kj5xr\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:43:48.630295 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.630055    2586 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-service-ca\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:43:48.630295 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.630065    2586 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0a688c1b-d943-4c51-b42f-b7c727b7ffc2-console-config\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:43:48.799671 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.799589    2586 generic.go:356] "Generic (PLEG): container finished" podID="0a688c1b-d943-4c51-b42f-b7c727b7ffc2" containerID="df8abb6ec2b86aad91bc89a6a856d044745b8ec23f6df7221079254e4b80db68" exitCode=0
Sep 17 16:43:48.799671 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.799637    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76795db869-sm9jc"
Sep 17 16:43:48.799671 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.799661    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76795db869-sm9jc" event={"ID":"0a688c1b-d943-4c51-b42f-b7c727b7ffc2","Type":"ContainerDied","Data":"df8abb6ec2b86aad91bc89a6a856d044745b8ec23f6df7221079254e4b80db68"}
Sep 17 16:43:48.799868 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.799702    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76795db869-sm9jc" event={"ID":"0a688c1b-d943-4c51-b42f-b7c727b7ffc2","Type":"ContainerDied","Data":"5fdfe94f890027b746458c6d36307561ca1cfb6d0824c36085f664ecc42920fa"}
Sep 17 16:43:48.799868 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.799721    2586 scope.go:117] "RemoveContainer" containerID="df8abb6ec2b86aad91bc89a6a856d044745b8ec23f6df7221079254e4b80db68"
Sep 17 16:43:48.808548 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.808529    2586 scope.go:117] "RemoveContainer" containerID="df8abb6ec2b86aad91bc89a6a856d044745b8ec23f6df7221079254e4b80db68"
Sep 17 16:43:48.808866 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:43:48.808847    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df8abb6ec2b86aad91bc89a6a856d044745b8ec23f6df7221079254e4b80db68\": container with ID starting with df8abb6ec2b86aad91bc89a6a856d044745b8ec23f6df7221079254e4b80db68 not found: ID does not exist" containerID="df8abb6ec2b86aad91bc89a6a856d044745b8ec23f6df7221079254e4b80db68"
Sep 17 16:43:48.808964 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.808900    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df8abb6ec2b86aad91bc89a6a856d044745b8ec23f6df7221079254e4b80db68"} err="failed to get container status \"df8abb6ec2b86aad91bc89a6a856d044745b8ec23f6df7221079254e4b80db68\": rpc error: code = NotFound desc = could not find container \"df8abb6ec2b86aad91bc89a6a856d044745b8ec23f6df7221079254e4b80db68\": container with ID starting with df8abb6ec2b86aad91bc89a6a856d044745b8ec23f6df7221079254e4b80db68 not found: ID does not exist"
Sep 17 16:43:48.820498 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.820465    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76795db869-sm9jc"]
Sep 17 16:43:48.826980 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:48.826955    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-76795db869-sm9jc"]
Sep 17 16:43:49.323620 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:49.323585    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a688c1b-d943-4c51-b42f-b7c727b7ffc2" path="/var/lib/kubelet/pods/0a688c1b-d943-4c51-b42f-b7c727b7ffc2/volumes"
Sep 17 16:43:54.711336 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:54.711301    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-nxc7t"
Sep 17 16:43:58.816461 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:58.816414    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6db46dc76b-cnrnc" podUID="ec225f41-f732-47ca-ac72-d92effb735b4" containerName="console" containerID="cri-o://74a078a4f5170134876286edde1b1debf065e9ec46e66648504ce1e087fb996d" gracePeriod=15
Sep 17 16:43:59.058530 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:59.058493    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6db46dc76b-cnrnc"
Sep 17 16:43:59.112714 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:59.112630    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec225f41-f732-47ca-ac72-d92effb735b4-oauth-serving-cert\") pod \"ec225f41-f732-47ca-ac72-d92effb735b4\" (UID: \"ec225f41-f732-47ca-ac72-d92effb735b4\") "
Sep 17 16:43:59.112714 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:59.112671    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec225f41-f732-47ca-ac72-d92effb735b4-console-config\") pod \"ec225f41-f732-47ca-ac72-d92effb735b4\" (UID: \"ec225f41-f732-47ca-ac72-d92effb735b4\") "
Sep 17 16:43:59.112714 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:59.112699    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec225f41-f732-47ca-ac72-d92effb735b4-console-serving-cert\") pod \"ec225f41-f732-47ca-ac72-d92effb735b4\" (UID: \"ec225f41-f732-47ca-ac72-d92effb735b4\") "
Sep 17 16:43:59.112714 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:59.112714    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec225f41-f732-47ca-ac72-d92effb735b4-service-ca\") pod \"ec225f41-f732-47ca-ac72-d92effb735b4\" (UID: \"ec225f41-f732-47ca-ac72-d92effb735b4\") "
Sep 17 16:43:59.113083 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:59.112845    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vlrb7\" (UniqueName: \"kubernetes.io/projected/ec225f41-f732-47ca-ac72-d92effb735b4-kube-api-access-vlrb7\") pod \"ec225f41-f732-47ca-ac72-d92effb735b4\" (UID: \"ec225f41-f732-47ca-ac72-d92effb735b4\") "
Sep 17 16:43:59.113083 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:59.112958    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec225f41-f732-47ca-ac72-d92effb735b4-console-oauth-config\") pod \"ec225f41-f732-47ca-ac72-d92effb735b4\" (UID: \"ec225f41-f732-47ca-ac72-d92effb735b4\") "
Sep 17 16:43:59.113179 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:59.113161    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec225f41-f732-47ca-ac72-d92effb735b4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ec225f41-f732-47ca-ac72-d92effb735b4" (UID: "ec225f41-f732-47ca-ac72-d92effb735b4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:43:59.113179 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:59.113166    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec225f41-f732-47ca-ac72-d92effb735b4-console-config" (OuterVolumeSpecName: "console-config") pod "ec225f41-f732-47ca-ac72-d92effb735b4" (UID: "ec225f41-f732-47ca-ac72-d92effb735b4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:43:59.113400 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:59.113375    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec225f41-f732-47ca-ac72-d92effb735b4-service-ca" (OuterVolumeSpecName: "service-ca") pod "ec225f41-f732-47ca-ac72-d92effb735b4" (UID: "ec225f41-f732-47ca-ac72-d92effb735b4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:43:59.115196 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:59.115173    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec225f41-f732-47ca-ac72-d92effb735b4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ec225f41-f732-47ca-ac72-d92effb735b4" (UID: "ec225f41-f732-47ca-ac72-d92effb735b4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:43:59.115283 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:59.115239    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec225f41-f732-47ca-ac72-d92effb735b4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ec225f41-f732-47ca-ac72-d92effb735b4" (UID: "ec225f41-f732-47ca-ac72-d92effb735b4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:43:59.115322 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:59.115288    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec225f41-f732-47ca-ac72-d92effb735b4-kube-api-access-vlrb7" (OuterVolumeSpecName: "kube-api-access-vlrb7") pod "ec225f41-f732-47ca-ac72-d92effb735b4" (UID: "ec225f41-f732-47ca-ac72-d92effb735b4"). InnerVolumeSpecName "kube-api-access-vlrb7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:43:59.213995 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:59.213951    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vlrb7\" (UniqueName: \"kubernetes.io/projected/ec225f41-f732-47ca-ac72-d92effb735b4-kube-api-access-vlrb7\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:43:59.213995 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:59.213985    2586 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec225f41-f732-47ca-ac72-d92effb735b4-console-oauth-config\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:43:59.213995 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:59.213996    2586 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec225f41-f732-47ca-ac72-d92effb735b4-oauth-serving-cert\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:43:59.213995 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:59.214006    2586 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec225f41-f732-47ca-ac72-d92effb735b4-console-config\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:43:59.214260 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:59.214017    2586 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec225f41-f732-47ca-ac72-d92effb735b4-console-serving-cert\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:43:59.214260 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:59.214025    2586 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec225f41-f732-47ca-ac72-d92effb735b4-service-ca\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:43:59.845221 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:59.845191    2586 generic.go:356] "Generic (PLEG): container finished" podID="ec225f41-f732-47ca-ac72-d92effb735b4" containerID="74a078a4f5170134876286edde1b1debf065e9ec46e66648504ce1e087fb996d" exitCode=0
Sep 17 16:43:59.845650 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:59.845240    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6db46dc76b-cnrnc"
Sep 17 16:43:59.845650 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:59.845279    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6db46dc76b-cnrnc" event={"ID":"ec225f41-f732-47ca-ac72-d92effb735b4","Type":"ContainerDied","Data":"74a078a4f5170134876286edde1b1debf065e9ec46e66648504ce1e087fb996d"}
Sep 17 16:43:59.845650 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:59.845337    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6db46dc76b-cnrnc" event={"ID":"ec225f41-f732-47ca-ac72-d92effb735b4","Type":"ContainerDied","Data":"e9018fe0f5c197cfedb2f762e0872a290b255f2bd39c9b2822f170dd9750f43f"}
Sep 17 16:43:59.845650 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:59.845364    2586 scope.go:117] "RemoveContainer" containerID="74a078a4f5170134876286edde1b1debf065e9ec46e66648504ce1e087fb996d"
Sep 17 16:43:59.853652 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:59.853633    2586 scope.go:117] "RemoveContainer" containerID="74a078a4f5170134876286edde1b1debf065e9ec46e66648504ce1e087fb996d"
Sep 17 16:43:59.853955 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:43:59.853934    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74a078a4f5170134876286edde1b1debf065e9ec46e66648504ce1e087fb996d\": container with ID starting with 74a078a4f5170134876286edde1b1debf065e9ec46e66648504ce1e087fb996d not found: ID does not exist" containerID="74a078a4f5170134876286edde1b1debf065e9ec46e66648504ce1e087fb996d"
Sep 17 16:43:59.854033 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:59.853962    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74a078a4f5170134876286edde1b1debf065e9ec46e66648504ce1e087fb996d"} err="failed to get container status \"74a078a4f5170134876286edde1b1debf065e9ec46e66648504ce1e087fb996d\": rpc error: code = NotFound desc = could not find container \"74a078a4f5170134876286edde1b1debf065e9ec46e66648504ce1e087fb996d\": container with ID starting with 74a078a4f5170134876286edde1b1debf065e9ec46e66648504ce1e087fb996d not found: ID does not exist"
Sep 17 16:43:59.862139 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:59.862115    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6db46dc76b-cnrnc"]
Sep 17 16:43:59.867619 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:43:59.864387    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6db46dc76b-cnrnc"]
Sep 17 16:44:01.322862 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:01.322813    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec225f41-f732-47ca-ac72-d92effb735b4" path="/var/lib/kubelet/pods/ec225f41-f732-47ca-ac72-d92effb735b4/volumes"
Sep 17 16:44:30.637138 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:30.637102    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 16:44:30.637626 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:30.637394    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerName="alertmanager" containerID="cri-o://2aaf886d418c93e4e4a047aff33a1bea5ab67054ad0f4debd83a4bf8360ee06d" gracePeriod=120
Sep 17 16:44:30.637626 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:30.637453    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerName="kube-rbac-proxy-metric" containerID="cri-o://ac59d045104ca38f10db38ed02966cd3b1d70945b4db47b4bbd71c53ad6a80b9" gracePeriod=120
Sep 17 16:44:30.637626 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:30.637479    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerName="kube-rbac-proxy-web" containerID="cri-o://65f556a0813ac886b5b9a4b3ee93aa42b881a7fa428146d0d4afa8136f234f64" gracePeriod=120
Sep 17 16:44:30.637626 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:30.637501    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerName="config-reloader" containerID="cri-o://9b65810bb5a87b5061386f85df47e0eca44d7e3d4796337968aca73a0b9059b1" gracePeriod=120
Sep 17 16:44:30.637626 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:30.637539    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerName="prom-label-proxy" containerID="cri-o://7fc8f3221de365777f83751714ce1c51d308115f62d8adaaf0a1504bca430850" gracePeriod=120
Sep 17 16:44:30.637626 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:30.637511    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerName="kube-rbac-proxy" containerID="cri-o://912ddc4deea35fef0a9f63c2375ecda6d3a1e0384afffabe7456fd5192de3b60" gracePeriod=120
Sep 17 16:44:30.958719 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:30.958638    2586 generic.go:356] "Generic (PLEG): container finished" podID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerID="7fc8f3221de365777f83751714ce1c51d308115f62d8adaaf0a1504bca430850" exitCode=0
Sep 17 16:44:30.958719 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:30.958665    2586 generic.go:356] "Generic (PLEG): container finished" podID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerID="912ddc4deea35fef0a9f63c2375ecda6d3a1e0384afffabe7456fd5192de3b60" exitCode=0
Sep 17 16:44:30.958719 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:30.958671    2586 generic.go:356] "Generic (PLEG): container finished" podID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerID="9b65810bb5a87b5061386f85df47e0eca44d7e3d4796337968aca73a0b9059b1" exitCode=0
Sep 17 16:44:30.958719 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:30.958677    2586 generic.go:356] "Generic (PLEG): container finished" podID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerID="2aaf886d418c93e4e4a047aff33a1bea5ab67054ad0f4debd83a4bf8360ee06d" exitCode=0
Sep 17 16:44:30.958719 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:30.958706    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a772efd-a0f1-4d09-afe7-72805b37ab2e","Type":"ContainerDied","Data":"7fc8f3221de365777f83751714ce1c51d308115f62d8adaaf0a1504bca430850"}
Sep 17 16:44:30.959002 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:30.958770    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a772efd-a0f1-4d09-afe7-72805b37ab2e","Type":"ContainerDied","Data":"912ddc4deea35fef0a9f63c2375ecda6d3a1e0384afffabe7456fd5192de3b60"}
Sep 17 16:44:30.959002 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:30.958804    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a772efd-a0f1-4d09-afe7-72805b37ab2e","Type":"ContainerDied","Data":"9b65810bb5a87b5061386f85df47e0eca44d7e3d4796337968aca73a0b9059b1"}
Sep 17 16:44:30.959002 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:30.958830    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a772efd-a0f1-4d09-afe7-72805b37ab2e","Type":"ContainerDied","Data":"2aaf886d418c93e4e4a047aff33a1bea5ab67054ad0f4debd83a4bf8360ee06d"}
Sep 17 16:44:31.879226 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.879202    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:31.962510 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.962416    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-secret-alertmanager-kube-rbac-proxy\") pod \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") "
Sep 17 16:44:31.962510 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.962453    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-config-volume\") pod \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") "
Sep 17 16:44:31.962510 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.962495    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-secret-alertmanager-kube-rbac-proxy-web\") pod \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") "
Sep 17 16:44:31.962773 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.962612    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-cluster-tls-config\") pod \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") "
Sep 17 16:44:31.962773 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.962676    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-web-config\") pod \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") "
Sep 17 16:44:31.962773 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.962712    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a772efd-a0f1-4d09-afe7-72805b37ab2e-tls-assets\") pod \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") "
Sep 17 16:44:31.962773 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.962738    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a772efd-a0f1-4d09-afe7-72805b37ab2e-alertmanager-trusted-ca-bundle\") pod \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") "
Sep 17 16:44:31.962965 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.962778    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4a772efd-a0f1-4d09-afe7-72805b37ab2e-alertmanager-main-db\") pod \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") "
Sep 17 16:44:31.962965 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.962803    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a772efd-a0f1-4d09-afe7-72805b37ab2e-metrics-client-ca\") pod \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") "
Sep 17 16:44:31.962965 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.962827    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2sr69\" (UniqueName: \"kubernetes.io/projected/4a772efd-a0f1-4d09-afe7-72805b37ab2e-kube-api-access-2sr69\") pod \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") "
Sep 17 16:44:31.962965 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.962862    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") "
Sep 17 16:44:31.962965 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.962931    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-secret-alertmanager-main-tls\") pod \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") "
Sep 17 16:44:31.962965 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.962958    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a772efd-a0f1-4d09-afe7-72805b37ab2e-config-out\") pod \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\" (UID: \"4a772efd-a0f1-4d09-afe7-72805b37ab2e\") "
Sep 17 16:44:31.963721 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.963691    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a772efd-a0f1-4d09-afe7-72805b37ab2e-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "4a772efd-a0f1-4d09-afe7-72805b37ab2e" (UID: "4a772efd-a0f1-4d09-afe7-72805b37ab2e"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 16:44:31.964402 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.964373    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a772efd-a0f1-4d09-afe7-72805b37ab2e-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "4a772efd-a0f1-4d09-afe7-72805b37ab2e" (UID: "4a772efd-a0f1-4d09-afe7-72805b37ab2e"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:44:31.965325 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.965042    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a772efd-a0f1-4d09-afe7-72805b37ab2e-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "4a772efd-a0f1-4d09-afe7-72805b37ab2e" (UID: "4a772efd-a0f1-4d09-afe7-72805b37ab2e"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:44:31.967032 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.966999    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "4a772efd-a0f1-4d09-afe7-72805b37ab2e" (UID: "4a772efd-a0f1-4d09-afe7-72805b37ab2e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:44:31.967567 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.967533    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "4a772efd-a0f1-4d09-afe7-72805b37ab2e" (UID: "4a772efd-a0f1-4d09-afe7-72805b37ab2e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:44:31.967679 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.967606    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a772efd-a0f1-4d09-afe7-72805b37ab2e-kube-api-access-2sr69" (OuterVolumeSpecName: "kube-api-access-2sr69") pod "4a772efd-a0f1-4d09-afe7-72805b37ab2e" (UID: "4a772efd-a0f1-4d09-afe7-72805b37ab2e"). InnerVolumeSpecName "kube-api-access-2sr69". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:44:31.968472 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.968181    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a772efd-a0f1-4d09-afe7-72805b37ab2e-config-out" (OuterVolumeSpecName: "config-out") pod "4a772efd-a0f1-4d09-afe7-72805b37ab2e" (UID: "4a772efd-a0f1-4d09-afe7-72805b37ab2e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 16:44:31.968472 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.968255    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-config-volume" (OuterVolumeSpecName: "config-volume") pod "4a772efd-a0f1-4d09-afe7-72805b37ab2e" (UID: "4a772efd-a0f1-4d09-afe7-72805b37ab2e"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:44:31.968472 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.968421    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "4a772efd-a0f1-4d09-afe7-72805b37ab2e" (UID: "4a772efd-a0f1-4d09-afe7-72805b37ab2e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:44:31.968956 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.968909    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a772efd-a0f1-4d09-afe7-72805b37ab2e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "4a772efd-a0f1-4d09-afe7-72805b37ab2e" (UID: "4a772efd-a0f1-4d09-afe7-72805b37ab2e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:44:31.968956 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.968933    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "4a772efd-a0f1-4d09-afe7-72805b37ab2e" (UID: "4a772efd-a0f1-4d09-afe7-72805b37ab2e"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:44:31.969968 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.969938    2586 generic.go:356] "Generic (PLEG): container finished" podID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerID="ac59d045104ca38f10db38ed02966cd3b1d70945b4db47b4bbd71c53ad6a80b9" exitCode=0
Sep 17 16:44:31.969968 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.969967    2586 generic.go:356] "Generic (PLEG): container finished" podID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerID="65f556a0813ac886b5b9a4b3ee93aa42b881a7fa428146d0d4afa8136f234f64" exitCode=0
Sep 17 16:44:31.970110 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.969993    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a772efd-a0f1-4d09-afe7-72805b37ab2e","Type":"ContainerDied","Data":"ac59d045104ca38f10db38ed02966cd3b1d70945b4db47b4bbd71c53ad6a80b9"}
Sep 17 16:44:31.970110 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.970056    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:31.970110 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.970081    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a772efd-a0f1-4d09-afe7-72805b37ab2e","Type":"ContainerDied","Data":"65f556a0813ac886b5b9a4b3ee93aa42b881a7fa428146d0d4afa8136f234f64"}
Sep 17 16:44:31.970255 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.970112    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a772efd-a0f1-4d09-afe7-72805b37ab2e","Type":"ContainerDied","Data":"ca0d18732794b587cc82a5a18f2ab91308d94d5897977e769d2b8b541043dc6c"}
Sep 17 16:44:31.970255 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.970151    2586 scope.go:117] "RemoveContainer" containerID="7fc8f3221de365777f83751714ce1c51d308115f62d8adaaf0a1504bca430850"
Sep 17 16:44:31.972701 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.972666    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "4a772efd-a0f1-4d09-afe7-72805b37ab2e" (UID: "4a772efd-a0f1-4d09-afe7-72805b37ab2e"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:44:31.979412 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.979385    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-web-config" (OuterVolumeSpecName: "web-config") pod "4a772efd-a0f1-4d09-afe7-72805b37ab2e" (UID: "4a772efd-a0f1-4d09-afe7-72805b37ab2e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:44:31.980211 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.980191    2586 scope.go:117] "RemoveContainer" containerID="ac59d045104ca38f10db38ed02966cd3b1d70945b4db47b4bbd71c53ad6a80b9"
Sep 17 16:44:31.986975 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.986959    2586 scope.go:117] "RemoveContainer" containerID="912ddc4deea35fef0a9f63c2375ecda6d3a1e0384afffabe7456fd5192de3b60"
Sep 17 16:44:31.993585 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.993569    2586 scope.go:117] "RemoveContainer" containerID="65f556a0813ac886b5b9a4b3ee93aa42b881a7fa428146d0d4afa8136f234f64"
Sep 17 16:44:31.999981 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:31.999965    2586 scope.go:117] "RemoveContainer" containerID="9b65810bb5a87b5061386f85df47e0eca44d7e3d4796337968aca73a0b9059b1"
Sep 17 16:44:32.006286 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.006271    2586 scope.go:117] "RemoveContainer" containerID="2aaf886d418c93e4e4a047aff33a1bea5ab67054ad0f4debd83a4bf8360ee06d"
Sep 17 16:44:32.012926 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.012909    2586 scope.go:117] "RemoveContainer" containerID="af0b4f3ac48be7f3d0482db5b50658413b00e5000801a7d93e41f85a3379c639"
Sep 17 16:44:32.019514 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.019497    2586 scope.go:117] "RemoveContainer" containerID="7fc8f3221de365777f83751714ce1c51d308115f62d8adaaf0a1504bca430850"
Sep 17 16:44:32.019781 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:44:32.019763    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fc8f3221de365777f83751714ce1c51d308115f62d8adaaf0a1504bca430850\": container with ID starting with 7fc8f3221de365777f83751714ce1c51d308115f62d8adaaf0a1504bca430850 not found: ID does not exist" containerID="7fc8f3221de365777f83751714ce1c51d308115f62d8adaaf0a1504bca430850"
Sep 17 16:44:32.019836 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.019791    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fc8f3221de365777f83751714ce1c51d308115f62d8adaaf0a1504bca430850"} err="failed to get container status \"7fc8f3221de365777f83751714ce1c51d308115f62d8adaaf0a1504bca430850\": rpc error: code = NotFound desc = could not find container \"7fc8f3221de365777f83751714ce1c51d308115f62d8adaaf0a1504bca430850\": container with ID starting with 7fc8f3221de365777f83751714ce1c51d308115f62d8adaaf0a1504bca430850 not found: ID does not exist"
Sep 17 16:44:32.019836 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.019809    2586 scope.go:117] "RemoveContainer" containerID="ac59d045104ca38f10db38ed02966cd3b1d70945b4db47b4bbd71c53ad6a80b9"
Sep 17 16:44:32.020098 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:44:32.020081    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac59d045104ca38f10db38ed02966cd3b1d70945b4db47b4bbd71c53ad6a80b9\": container with ID starting with ac59d045104ca38f10db38ed02966cd3b1d70945b4db47b4bbd71c53ad6a80b9 not found: ID does not exist" containerID="ac59d045104ca38f10db38ed02966cd3b1d70945b4db47b4bbd71c53ad6a80b9"
Sep 17 16:44:32.020152 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.020104    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac59d045104ca38f10db38ed02966cd3b1d70945b4db47b4bbd71c53ad6a80b9"} err="failed to get container status \"ac59d045104ca38f10db38ed02966cd3b1d70945b4db47b4bbd71c53ad6a80b9\": rpc error: code = NotFound desc = could not find container \"ac59d045104ca38f10db38ed02966cd3b1d70945b4db47b4bbd71c53ad6a80b9\": container with ID starting with ac59d045104ca38f10db38ed02966cd3b1d70945b4db47b4bbd71c53ad6a80b9 not found: ID does not exist"
Sep 17 16:44:32.020152 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.020121    2586 scope.go:117] "RemoveContainer" containerID="912ddc4deea35fef0a9f63c2375ecda6d3a1e0384afffabe7456fd5192de3b60"
Sep 17 16:44:32.020372 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:44:32.020357    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"912ddc4deea35fef0a9f63c2375ecda6d3a1e0384afffabe7456fd5192de3b60\": container with ID starting with 912ddc4deea35fef0a9f63c2375ecda6d3a1e0384afffabe7456fd5192de3b60 not found: ID does not exist" containerID="912ddc4deea35fef0a9f63c2375ecda6d3a1e0384afffabe7456fd5192de3b60"
Sep 17 16:44:32.020414 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.020377    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"912ddc4deea35fef0a9f63c2375ecda6d3a1e0384afffabe7456fd5192de3b60"} err="failed to get container status \"912ddc4deea35fef0a9f63c2375ecda6d3a1e0384afffabe7456fd5192de3b60\": rpc error: code = NotFound desc = could not find container \"912ddc4deea35fef0a9f63c2375ecda6d3a1e0384afffabe7456fd5192de3b60\": container with ID starting with 912ddc4deea35fef0a9f63c2375ecda6d3a1e0384afffabe7456fd5192de3b60 not found: ID does not exist"
Sep 17 16:44:32.020414 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.020391    2586 scope.go:117] "RemoveContainer" containerID="65f556a0813ac886b5b9a4b3ee93aa42b881a7fa428146d0d4afa8136f234f64"
Sep 17 16:44:32.020641 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:44:32.020625    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65f556a0813ac886b5b9a4b3ee93aa42b881a7fa428146d0d4afa8136f234f64\": container with ID starting with 65f556a0813ac886b5b9a4b3ee93aa42b881a7fa428146d0d4afa8136f234f64 not found: ID does not exist" containerID="65f556a0813ac886b5b9a4b3ee93aa42b881a7fa428146d0d4afa8136f234f64"
Sep 17 16:44:32.020699 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.020644    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65f556a0813ac886b5b9a4b3ee93aa42b881a7fa428146d0d4afa8136f234f64"} err="failed to get container status \"65f556a0813ac886b5b9a4b3ee93aa42b881a7fa428146d0d4afa8136f234f64\": rpc error: code = NotFound desc = could not find container \"65f556a0813ac886b5b9a4b3ee93aa42b881a7fa428146d0d4afa8136f234f64\": container with ID starting with 65f556a0813ac886b5b9a4b3ee93aa42b881a7fa428146d0d4afa8136f234f64 not found: ID does not exist"
Sep 17 16:44:32.020699 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.020657    2586 scope.go:117] "RemoveContainer" containerID="9b65810bb5a87b5061386f85df47e0eca44d7e3d4796337968aca73a0b9059b1"
Sep 17 16:44:32.020847 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:44:32.020831    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b65810bb5a87b5061386f85df47e0eca44d7e3d4796337968aca73a0b9059b1\": container with ID starting with 9b65810bb5a87b5061386f85df47e0eca44d7e3d4796337968aca73a0b9059b1 not found: ID does not exist" containerID="9b65810bb5a87b5061386f85df47e0eca44d7e3d4796337968aca73a0b9059b1"
Sep 17 16:44:32.020909 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.020850    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b65810bb5a87b5061386f85df47e0eca44d7e3d4796337968aca73a0b9059b1"} err="failed to get container status \"9b65810bb5a87b5061386f85df47e0eca44d7e3d4796337968aca73a0b9059b1\": rpc error: code = NotFound desc = could not find container \"9b65810bb5a87b5061386f85df47e0eca44d7e3d4796337968aca73a0b9059b1\": container with ID starting with 9b65810bb5a87b5061386f85df47e0eca44d7e3d4796337968aca73a0b9059b1 not found: ID does not exist"
Sep 17 16:44:32.020909 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.020862    2586 scope.go:117] "RemoveContainer" containerID="2aaf886d418c93e4e4a047aff33a1bea5ab67054ad0f4debd83a4bf8360ee06d"
Sep 17 16:44:32.021111 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:44:32.021097    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2aaf886d418c93e4e4a047aff33a1bea5ab67054ad0f4debd83a4bf8360ee06d\": container with ID starting with 2aaf886d418c93e4e4a047aff33a1bea5ab67054ad0f4debd83a4bf8360ee06d not found: ID does not exist" containerID="2aaf886d418c93e4e4a047aff33a1bea5ab67054ad0f4debd83a4bf8360ee06d"
Sep 17 16:44:32.021151 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.021114    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2aaf886d418c93e4e4a047aff33a1bea5ab67054ad0f4debd83a4bf8360ee06d"} err="failed to get container status \"2aaf886d418c93e4e4a047aff33a1bea5ab67054ad0f4debd83a4bf8360ee06d\": rpc error: code = NotFound desc = could not find container \"2aaf886d418c93e4e4a047aff33a1bea5ab67054ad0f4debd83a4bf8360ee06d\": container with ID starting with 2aaf886d418c93e4e4a047aff33a1bea5ab67054ad0f4debd83a4bf8360ee06d not found: ID does not exist"
Sep 17 16:44:32.021151 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.021126    2586 scope.go:117] "RemoveContainer" containerID="af0b4f3ac48be7f3d0482db5b50658413b00e5000801a7d93e41f85a3379c639"
Sep 17 16:44:32.021373 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:44:32.021356    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af0b4f3ac48be7f3d0482db5b50658413b00e5000801a7d93e41f85a3379c639\": container with ID starting with af0b4f3ac48be7f3d0482db5b50658413b00e5000801a7d93e41f85a3379c639 not found: ID does not exist" containerID="af0b4f3ac48be7f3d0482db5b50658413b00e5000801a7d93e41f85a3379c639"
Sep 17 16:44:32.021420 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.021377    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af0b4f3ac48be7f3d0482db5b50658413b00e5000801a7d93e41f85a3379c639"} err="failed to get container status \"af0b4f3ac48be7f3d0482db5b50658413b00e5000801a7d93e41f85a3379c639\": rpc error: code = NotFound desc = could not find container \"af0b4f3ac48be7f3d0482db5b50658413b00e5000801a7d93e41f85a3379c639\": container with ID starting with af0b4f3ac48be7f3d0482db5b50658413b00e5000801a7d93e41f85a3379c639 not found: ID does not exist"
Sep 17 16:44:32.021420 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.021390    2586 scope.go:117] "RemoveContainer" containerID="7fc8f3221de365777f83751714ce1c51d308115f62d8adaaf0a1504bca430850"
Sep 17 16:44:32.021601 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.021586    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fc8f3221de365777f83751714ce1c51d308115f62d8adaaf0a1504bca430850"} err="failed to get container status \"7fc8f3221de365777f83751714ce1c51d308115f62d8adaaf0a1504bca430850\": rpc error: code = NotFound desc = could not find container \"7fc8f3221de365777f83751714ce1c51d308115f62d8adaaf0a1504bca430850\": container with ID starting with 7fc8f3221de365777f83751714ce1c51d308115f62d8adaaf0a1504bca430850 not found: ID does not exist"
Sep 17 16:44:32.021689 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.021602    2586 scope.go:117] "RemoveContainer" containerID="ac59d045104ca38f10db38ed02966cd3b1d70945b4db47b4bbd71c53ad6a80b9"
Sep 17 16:44:32.021830 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.021815    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac59d045104ca38f10db38ed02966cd3b1d70945b4db47b4bbd71c53ad6a80b9"} err="failed to get container status \"ac59d045104ca38f10db38ed02966cd3b1d70945b4db47b4bbd71c53ad6a80b9\": rpc error: code = NotFound desc = could not find container \"ac59d045104ca38f10db38ed02966cd3b1d70945b4db47b4bbd71c53ad6a80b9\": container with ID starting with ac59d045104ca38f10db38ed02966cd3b1d70945b4db47b4bbd71c53ad6a80b9 not found: ID does not exist"
Sep 17 16:44:32.021894 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.021830    2586 scope.go:117] "RemoveContainer" containerID="912ddc4deea35fef0a9f63c2375ecda6d3a1e0384afffabe7456fd5192de3b60"
Sep 17 16:44:32.022081 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.022062    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"912ddc4deea35fef0a9f63c2375ecda6d3a1e0384afffabe7456fd5192de3b60"} err="failed to get container status \"912ddc4deea35fef0a9f63c2375ecda6d3a1e0384afffabe7456fd5192de3b60\": rpc error: code = NotFound desc = could not find container \"912ddc4deea35fef0a9f63c2375ecda6d3a1e0384afffabe7456fd5192de3b60\": container with ID starting with 912ddc4deea35fef0a9f63c2375ecda6d3a1e0384afffabe7456fd5192de3b60 not found: ID does not exist"
Sep 17 16:44:32.022131 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.022082    2586 scope.go:117] "RemoveContainer" containerID="65f556a0813ac886b5b9a4b3ee93aa42b881a7fa428146d0d4afa8136f234f64"
Sep 17 16:44:32.022290 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.022275    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65f556a0813ac886b5b9a4b3ee93aa42b881a7fa428146d0d4afa8136f234f64"} err="failed to get container status \"65f556a0813ac886b5b9a4b3ee93aa42b881a7fa428146d0d4afa8136f234f64\": rpc error: code = NotFound desc = could not find container \"65f556a0813ac886b5b9a4b3ee93aa42b881a7fa428146d0d4afa8136f234f64\": container with ID starting with 65f556a0813ac886b5b9a4b3ee93aa42b881a7fa428146d0d4afa8136f234f64 not found: ID does not exist"
Sep 17 16:44:32.022335 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.022290    2586 scope.go:117] "RemoveContainer" containerID="9b65810bb5a87b5061386f85df47e0eca44d7e3d4796337968aca73a0b9059b1"
Sep 17 16:44:32.022482 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.022466    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b65810bb5a87b5061386f85df47e0eca44d7e3d4796337968aca73a0b9059b1"} err="failed to get container status \"9b65810bb5a87b5061386f85df47e0eca44d7e3d4796337968aca73a0b9059b1\": rpc error: code = NotFound desc = could not find container \"9b65810bb5a87b5061386f85df47e0eca44d7e3d4796337968aca73a0b9059b1\": container with ID starting with 9b65810bb5a87b5061386f85df47e0eca44d7e3d4796337968aca73a0b9059b1 not found: ID does not exist"
Sep 17 16:44:32.022523 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.022484    2586 scope.go:117] "RemoveContainer" containerID="2aaf886d418c93e4e4a047aff33a1bea5ab67054ad0f4debd83a4bf8360ee06d"
Sep 17 16:44:32.022682 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.022667    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2aaf886d418c93e4e4a047aff33a1bea5ab67054ad0f4debd83a4bf8360ee06d"} err="failed to get container status \"2aaf886d418c93e4e4a047aff33a1bea5ab67054ad0f4debd83a4bf8360ee06d\": rpc error: code = NotFound desc = could not find container \"2aaf886d418c93e4e4a047aff33a1bea5ab67054ad0f4debd83a4bf8360ee06d\": container with ID starting with 2aaf886d418c93e4e4a047aff33a1bea5ab67054ad0f4debd83a4bf8360ee06d not found: ID does not exist"
Sep 17 16:44:32.022722 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.022683    2586 scope.go:117] "RemoveContainer" containerID="af0b4f3ac48be7f3d0482db5b50658413b00e5000801a7d93e41f85a3379c639"
Sep 17 16:44:32.022942 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.022856    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af0b4f3ac48be7f3d0482db5b50658413b00e5000801a7d93e41f85a3379c639"} err="failed to get container status \"af0b4f3ac48be7f3d0482db5b50658413b00e5000801a7d93e41f85a3379c639\": rpc error: code = NotFound desc = could not find container \"af0b4f3ac48be7f3d0482db5b50658413b00e5000801a7d93e41f85a3379c639\": container with ID starting with af0b4f3ac48be7f3d0482db5b50658413b00e5000801a7d93e41f85a3379c639 not found: ID does not exist"
Sep 17 16:44:32.064256 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.064222    2586 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-secret-alertmanager-main-tls\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:44:32.064256 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.064251    2586 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a772efd-a0f1-4d09-afe7-72805b37ab2e-config-out\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:44:32.064256 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.064262    2586 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:44:32.064423 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.064270    2586 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-config-volume\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:44:32.064423 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.064280    2586 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:44:32.064423 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.064288    2586 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-cluster-tls-config\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:44:32.064423 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.064296    2586 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-web-config\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:44:32.064423 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.064304    2586 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a772efd-a0f1-4d09-afe7-72805b37ab2e-tls-assets\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:44:32.064423 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.064314    2586 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a772efd-a0f1-4d09-afe7-72805b37ab2e-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:44:32.064423 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.064322    2586 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4a772efd-a0f1-4d09-afe7-72805b37ab2e-alertmanager-main-db\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:44:32.064423 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.064330    2586 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a772efd-a0f1-4d09-afe7-72805b37ab2e-metrics-client-ca\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:44:32.064423 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.064338    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2sr69\" (UniqueName: \"kubernetes.io/projected/4a772efd-a0f1-4d09-afe7-72805b37ab2e-kube-api-access-2sr69\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:44:32.064423 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.064346    2586 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4a772efd-a0f1-4d09-afe7-72805b37ab2e-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:44:32.297337 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.297306    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 16:44:32.303338 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.303304    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 16:44:32.343211 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.343180    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 16:44:32.343434 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.343422    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerName="kube-rbac-proxy-metric"
Sep 17 16:44:32.343474 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.343436    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerName="kube-rbac-proxy-metric"
Sep 17 16:44:32.343474 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.343444    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerName="prom-label-proxy"
Sep 17 16:44:32.343474 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.343449    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerName="prom-label-proxy"
Sep 17 16:44:32.343474 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.343457    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerName="init-config-reloader"
Sep 17 16:44:32.343474 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.343462    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerName="init-config-reloader"
Sep 17 16:44:32.343474 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.343469    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a688c1b-d943-4c51-b42f-b7c727b7ffc2" containerName="console"
Sep 17 16:44:32.343474 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.343475    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a688c1b-d943-4c51-b42f-b7c727b7ffc2" containerName="console"
Sep 17 16:44:32.343667 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.343482    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerName="kube-rbac-proxy-web"
Sep 17 16:44:32.343667 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.343487    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerName="kube-rbac-proxy-web"
Sep 17 16:44:32.343667 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.343497    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerName="kube-rbac-proxy"
Sep 17 16:44:32.343667 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.343501    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerName="kube-rbac-proxy"
Sep 17 16:44:32.343667 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.343510    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec225f41-f732-47ca-ac72-d92effb735b4" containerName="console"
Sep 17 16:44:32.343667 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.343515    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec225f41-f732-47ca-ac72-d92effb735b4" containerName="console"
Sep 17 16:44:32.343667 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.343521    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerName="alertmanager"
Sep 17 16:44:32.343667 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.343526    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerName="alertmanager"
Sep 17 16:44:32.343667 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.343530    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerName="config-reloader"
Sep 17 16:44:32.343667 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.343535    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerName="config-reloader"
Sep 17 16:44:32.343667 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.343571    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a688c1b-d943-4c51-b42f-b7c727b7ffc2" containerName="console"
Sep 17 16:44:32.343667 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.343577    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec225f41-f732-47ca-ac72-d92effb735b4" containerName="console"
Sep 17 16:44:32.343667 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.343583    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerName="kube-rbac-proxy-metric"
Sep 17 16:44:32.343667 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.343588    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerName="config-reloader"
Sep 17 16:44:32.343667 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.343593    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerName="kube-rbac-proxy-web"
Sep 17 16:44:32.343667 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.343599    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerName="prom-label-proxy"
Sep 17 16:44:32.343667 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.343604    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerName="kube-rbac-proxy"
Sep 17 16:44:32.343667 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.343610    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" containerName="alertmanager"
Sep 17 16:44:32.348558 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.348539    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.351391 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.351366    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 17 16:44:32.351552 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.351512    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-wprm8\""
Sep 17 16:44:32.351664 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.351558    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 17 16:44:32.351852 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.351831    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 17 16:44:32.351988 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.351889    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 17 16:44:32.351988 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.351899    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 17 16:44:32.351988 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.351927    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 17 16:44:32.351988 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.351830    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 17 16:44:32.352181 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.351935    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 17 16:44:32.358634 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.358618    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 17 16:44:32.364975 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.364956    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 16:44:32.467855 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.467816    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.467855 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.467854    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-web-config\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.468081 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.467916    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-config-volume\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.468081 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.467943    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.468081 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.467962    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvd8d\" (UniqueName: \"kubernetes.io/projected/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-kube-api-access-gvd8d\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.468081 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.467976    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.468081 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.467991    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.468081 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.468055    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.468258 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.468108    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.468258 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.468126    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.468258 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.468157    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.468258 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.468195    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.468258 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.468215    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-config-out\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.568934 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.568815    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-config-volume\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.568934 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.568865    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.568934 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.568917    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvd8d\" (UniqueName: \"kubernetes.io/projected/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-kube-api-access-gvd8d\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.569190 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.568943    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.569190 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.569118    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.569190 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.569156    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.569340 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.569248    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.569340 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.569306    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.569447 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.569356    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.569447 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.569410    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.569557 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.569477    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-config-out\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.569557 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.569510    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.569653 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.569554    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-web-config\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.570678 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.570320    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.570678 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.570373    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.570678 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.570436    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.572398 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.572370    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.572503 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.572416    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.572569 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.572515    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-config-volume\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.572755 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.572727    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-config-out\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.572913 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.572842    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.573114 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.573096    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.573199 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.573184    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-web-config\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.573899 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.573849    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.574439 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.574419    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.577139 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.577117    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvd8d\" (UniqueName: \"kubernetes.io/projected/df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3-kube-api-access-gvd8d\") pod \"alertmanager-main-0\" (UID: \"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.658341 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.658304    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 16:44:32.942790 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.942584    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 16:44:32.946544 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:44:32.946519    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf2f7bfc_ac59_46ec_8e2a_bbef2de4b6e3.slice/crio-03ad55ac63293e78e13e0c47b1ec95f08daf671d678a9ab652d4cc36e6ff8e3d WatchSource:0}: Error finding container 03ad55ac63293e78e13e0c47b1ec95f08daf671d678a9ab652d4cc36e6ff8e3d: Status 404 returned error can't find the container with id 03ad55ac63293e78e13e0c47b1ec95f08daf671d678a9ab652d4cc36e6ff8e3d
Sep 17 16:44:32.974684 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:32.974654    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3","Type":"ContainerStarted","Data":"03ad55ac63293e78e13e0c47b1ec95f08daf671d678a9ab652d4cc36e6ff8e3d"}
Sep 17 16:44:33.323523 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:33.323494    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a772efd-a0f1-4d09-afe7-72805b37ab2e" path="/var/lib/kubelet/pods/4a772efd-a0f1-4d09-afe7-72805b37ab2e/volumes"
Sep 17 16:44:33.980327 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:33.980242    2586 generic.go:356] "Generic (PLEG): container finished" podID="df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3" containerID="c0ad0300341b4a8be0abec16792b450d7aa2e897fb9854c968dda967224c4313" exitCode=0
Sep 17 16:44:33.980327 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:33.980287    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3","Type":"ContainerDied","Data":"c0ad0300341b4a8be0abec16792b450d7aa2e897fb9854c968dda967224c4313"}
Sep 17 16:44:34.668038 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.667998    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-869c579c6-cclth"]
Sep 17 16:44:34.671685 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.671664    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-869c579c6-cclth"
Sep 17 16:44:34.676376 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.676352    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 17 16:44:34.676376 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.676352    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 17 16:44:34.676541 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.676362    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 17 16:44:34.676702 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.676677    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 17 16:44:34.676825 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.676805    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-d68hq\""
Sep 17 16:44:34.676926 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.676838    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 17 16:44:34.685613 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.685591    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 17 16:44:34.700982 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.700956    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-869c579c6-cclth"]
Sep 17 16:44:34.787780 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.787742    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22bc52f8-271f-4582-af20-40590e7f9935-serving-certs-ca-bundle\") pod \"telemeter-client-869c579c6-cclth\" (UID: \"22bc52f8-271f-4582-af20-40590e7f9935\") " pod="openshift-monitoring/telemeter-client-869c579c6-cclth"
Sep 17 16:44:34.787780 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.787797    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22bc52f8-271f-4582-af20-40590e7f9935-telemeter-trusted-ca-bundle\") pod \"telemeter-client-869c579c6-cclth\" (UID: \"22bc52f8-271f-4582-af20-40590e7f9935\") " pod="openshift-monitoring/telemeter-client-869c579c6-cclth"
Sep 17 16:44:34.788071 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.787820    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/22bc52f8-271f-4582-af20-40590e7f9935-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-869c579c6-cclth\" (UID: \"22bc52f8-271f-4582-af20-40590e7f9935\") " pod="openshift-monitoring/telemeter-client-869c579c6-cclth"
Sep 17 16:44:34.788071 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.787934    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbrq6\" (UniqueName: \"kubernetes.io/projected/22bc52f8-271f-4582-af20-40590e7f9935-kube-api-access-zbrq6\") pod \"telemeter-client-869c579c6-cclth\" (UID: \"22bc52f8-271f-4582-af20-40590e7f9935\") " pod="openshift-monitoring/telemeter-client-869c579c6-cclth"
Sep 17 16:44:34.788071 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.787962    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/22bc52f8-271f-4582-af20-40590e7f9935-secret-telemeter-client\") pod \"telemeter-client-869c579c6-cclth\" (UID: \"22bc52f8-271f-4582-af20-40590e7f9935\") " pod="openshift-monitoring/telemeter-client-869c579c6-cclth"
Sep 17 16:44:34.788071 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.787980    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/22bc52f8-271f-4582-af20-40590e7f9935-telemeter-client-tls\") pod \"telemeter-client-869c579c6-cclth\" (UID: \"22bc52f8-271f-4582-af20-40590e7f9935\") " pod="openshift-monitoring/telemeter-client-869c579c6-cclth"
Sep 17 16:44:34.788071 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.788050    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/22bc52f8-271f-4582-af20-40590e7f9935-federate-client-tls\") pod \"telemeter-client-869c579c6-cclth\" (UID: \"22bc52f8-271f-4582-af20-40590e7f9935\") " pod="openshift-monitoring/telemeter-client-869c579c6-cclth"
Sep 17 16:44:34.788315 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.788082    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22bc52f8-271f-4582-af20-40590e7f9935-metrics-client-ca\") pod \"telemeter-client-869c579c6-cclth\" (UID: \"22bc52f8-271f-4582-af20-40590e7f9935\") " pod="openshift-monitoring/telemeter-client-869c579c6-cclth"
Sep 17 16:44:34.889231 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.889195    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22bc52f8-271f-4582-af20-40590e7f9935-serving-certs-ca-bundle\") pod \"telemeter-client-869c579c6-cclth\" (UID: \"22bc52f8-271f-4582-af20-40590e7f9935\") " pod="openshift-monitoring/telemeter-client-869c579c6-cclth"
Sep 17 16:44:34.889432 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.889243    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22bc52f8-271f-4582-af20-40590e7f9935-telemeter-trusted-ca-bundle\") pod \"telemeter-client-869c579c6-cclth\" (UID: \"22bc52f8-271f-4582-af20-40590e7f9935\") " pod="openshift-monitoring/telemeter-client-869c579c6-cclth"
Sep 17 16:44:34.889432 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.889268    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/22bc52f8-271f-4582-af20-40590e7f9935-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-869c579c6-cclth\" (UID: \"22bc52f8-271f-4582-af20-40590e7f9935\") " pod="openshift-monitoring/telemeter-client-869c579c6-cclth"
Sep 17 16:44:34.889432 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.889295    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbrq6\" (UniqueName: \"kubernetes.io/projected/22bc52f8-271f-4582-af20-40590e7f9935-kube-api-access-zbrq6\") pod \"telemeter-client-869c579c6-cclth\" (UID: \"22bc52f8-271f-4582-af20-40590e7f9935\") " pod="openshift-monitoring/telemeter-client-869c579c6-cclth"
Sep 17 16:44:34.889432 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.889311    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/22bc52f8-271f-4582-af20-40590e7f9935-secret-telemeter-client\") pod \"telemeter-client-869c579c6-cclth\" (UID: \"22bc52f8-271f-4582-af20-40590e7f9935\") " pod="openshift-monitoring/telemeter-client-869c579c6-cclth"
Sep 17 16:44:34.889432 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.889326    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/22bc52f8-271f-4582-af20-40590e7f9935-telemeter-client-tls\") pod \"telemeter-client-869c579c6-cclth\" (UID: \"22bc52f8-271f-4582-af20-40590e7f9935\") " pod="openshift-monitoring/telemeter-client-869c579c6-cclth"
Sep 17 16:44:34.889432 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.889347    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/22bc52f8-271f-4582-af20-40590e7f9935-federate-client-tls\") pod \"telemeter-client-869c579c6-cclth\" (UID: \"22bc52f8-271f-4582-af20-40590e7f9935\") " pod="openshift-monitoring/telemeter-client-869c579c6-cclth"
Sep 17 16:44:34.889432 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.889362    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22bc52f8-271f-4582-af20-40590e7f9935-metrics-client-ca\") pod \"telemeter-client-869c579c6-cclth\" (UID: \"22bc52f8-271f-4582-af20-40590e7f9935\") " pod="openshift-monitoring/telemeter-client-869c579c6-cclth"
Sep 17 16:44:34.890102 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.890065    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22bc52f8-271f-4582-af20-40590e7f9935-serving-certs-ca-bundle\") pod \"telemeter-client-869c579c6-cclth\" (UID: \"22bc52f8-271f-4582-af20-40590e7f9935\") " pod="openshift-monitoring/telemeter-client-869c579c6-cclth"
Sep 17 16:44:34.890217 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.890149    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22bc52f8-271f-4582-af20-40590e7f9935-metrics-client-ca\") pod \"telemeter-client-869c579c6-cclth\" (UID: \"22bc52f8-271f-4582-af20-40590e7f9935\") " pod="openshift-monitoring/telemeter-client-869c579c6-cclth"
Sep 17 16:44:34.890645 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.890622    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22bc52f8-271f-4582-af20-40590e7f9935-telemeter-trusted-ca-bundle\") pod \"telemeter-client-869c579c6-cclth\" (UID: \"22bc52f8-271f-4582-af20-40590e7f9935\") " pod="openshift-monitoring/telemeter-client-869c579c6-cclth"
Sep 17 16:44:34.892106 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.892078    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/22bc52f8-271f-4582-af20-40590e7f9935-telemeter-client-tls\") pod \"telemeter-client-869c579c6-cclth\" (UID: \"22bc52f8-271f-4582-af20-40590e7f9935\") " pod="openshift-monitoring/telemeter-client-869c579c6-cclth"
Sep 17 16:44:34.892311 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.892291    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/22bc52f8-271f-4582-af20-40590e7f9935-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-869c579c6-cclth\" (UID: \"22bc52f8-271f-4582-af20-40590e7f9935\") " pod="openshift-monitoring/telemeter-client-869c579c6-cclth"
Sep 17 16:44:34.892764 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.892741    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/22bc52f8-271f-4582-af20-40590e7f9935-secret-telemeter-client\") pod \"telemeter-client-869c579c6-cclth\" (UID: \"22bc52f8-271f-4582-af20-40590e7f9935\") " pod="openshift-monitoring/telemeter-client-869c579c6-cclth"
Sep 17 16:44:34.892764 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.892759    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/22bc52f8-271f-4582-af20-40590e7f9935-federate-client-tls\") pod \"telemeter-client-869c579c6-cclth\" (UID: \"22bc52f8-271f-4582-af20-40590e7f9935\") " pod="openshift-monitoring/telemeter-client-869c579c6-cclth"
Sep 17 16:44:34.897150 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.897130    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbrq6\" (UniqueName: \"kubernetes.io/projected/22bc52f8-271f-4582-af20-40590e7f9935-kube-api-access-zbrq6\") pod \"telemeter-client-869c579c6-cclth\" (UID: \"22bc52f8-271f-4582-af20-40590e7f9935\") " pod="openshift-monitoring/telemeter-client-869c579c6-cclth"
Sep 17 16:44:34.983453 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.983354    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-869c579c6-cclth"
Sep 17 16:44:34.989217 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.989191    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3","Type":"ContainerStarted","Data":"d26c9c2d26a7e9afd4c57feaf8db498485e3cea315a218c5903d599b27d1981c"}
Sep 17 16:44:34.989330 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.989251    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3","Type":"ContainerStarted","Data":"e94492ff968c35a033887be05b9a8049c49f1ce7c2128ef7271185201ebdbfa1"}
Sep 17 16:44:34.989330 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.989279    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3","Type":"ContainerStarted","Data":"5978533f1f322a3d1be2260f38f51daf03ea1152beed6e3f14f9ba89fb10ce2d"}
Sep 17 16:44:34.989330 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.989302    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3","Type":"ContainerStarted","Data":"e95f6f33911f00298402a3fc424b4e9c74f1f3c2f6160812077ae6501977e853"}
Sep 17 16:44:34.989330 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.989325    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3","Type":"ContainerStarted","Data":"060772e7ac95beed6facb3e655346b0c2b5e74a19bc525a32e550159b9d25678"}
Sep 17 16:44:34.989514 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:34.989351    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3","Type":"ContainerStarted","Data":"eeb0c88482dcdac33bb808b11784f2ef4e5ec022455f58bf86ea1c9dbfd3d87d"}
Sep 17 16:44:35.018671 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:35.017578    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.000974117 podStartE2EDuration="3.017552098s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.577981ms" isStatefulPod=false podCreationTimestamp="2026-09-17 16:44:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:44:35.01707334 +0000 UTC m=+145.718152003" watchObservedRunningTime="2026-09-17 16:44:35.017552098 +0000 UTC m=+145.718630762"
Sep 17 16:44:35.269758 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:35.269732    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-869c579c6-cclth"]
Sep 17 16:44:35.272034 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:44:35.271990    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22bc52f8_271f_4582_af20_40590e7f9935.slice/crio-ed6c4c797c6ab263592399b939455b515b04a68d5c3e1b6e54b8072d51e93870 WatchSource:0}: Error finding container ed6c4c797c6ab263592399b939455b515b04a68d5c3e1b6e54b8072d51e93870: Status 404 returned error can't find the container with id ed6c4c797c6ab263592399b939455b515b04a68d5c3e1b6e54b8072d51e93870
Sep 17 16:44:35.996242 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:35.996199    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-869c579c6-cclth" event={"ID":"22bc52f8-271f-4582-af20-40590e7f9935","Type":"ContainerStarted","Data":"ed6c4c797c6ab263592399b939455b515b04a68d5c3e1b6e54b8072d51e93870"}
Sep 17 16:44:38.006475 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:38.006435    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-869c579c6-cclth" event={"ID":"22bc52f8-271f-4582-af20-40590e7f9935","Type":"ContainerStarted","Data":"179027dc75ccd9ccafa3bdd9cc5c8fe6983c05632f8e97ca80f36b2927623357"}
Sep 17 16:44:38.006970 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:38.006503    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-869c579c6-cclth" event={"ID":"22bc52f8-271f-4582-af20-40590e7f9935","Type":"ContainerStarted","Data":"4a08e5aef836d7ce68ff9f6329acb320ee48bfa26b5c2c90208702790e27162a"}
Sep 17 16:44:38.006970 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:38.006529    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-869c579c6-cclth" event={"ID":"22bc52f8-271f-4582-af20-40590e7f9935","Type":"ContainerStarted","Data":"cf2e1ac980d4e8b8b507d68c245f82b0c7f638a33779b958b116992b732aba7b"}
Sep 17 16:44:38.028284 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:38.028227    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-869c579c6-cclth" podStartSLOduration=2.325010124 podStartE2EDuration="4.028212574s" totalImagesPullingTime="1.70320245s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:44:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:44:38.02736347 +0000 UTC m=+148.728442132" watchObservedRunningTime="2026-09-17 16:44:38.028212574 +0000 UTC m=+148.729291249"
Sep 17 16:44:38.112790 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:38.112752    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-59645bbb45-msj8g"]
Sep 17 16:44:38.116009 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:38.115980    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59645bbb45-msj8g"
Sep 17 16:44:38.126828 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:38.126805    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59645bbb45-msj8g"]
Sep 17 16:44:38.216576 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:38.216536    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd1becfb-1d2f-4670-8f53-516caebc4fb4-trusted-ca-bundle\") pod \"console-59645bbb45-msj8g\" (UID: \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\") " pod="openshift-console/console-59645bbb45-msj8g"
Sep 17 16:44:38.216576 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:38.216575    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd1becfb-1d2f-4670-8f53-516caebc4fb4-oauth-serving-cert\") pod \"console-59645bbb45-msj8g\" (UID: \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\") " pod="openshift-console/console-59645bbb45-msj8g"
Sep 17 16:44:38.216813 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:38.216603    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd1becfb-1d2f-4670-8f53-516caebc4fb4-console-oauth-config\") pod \"console-59645bbb45-msj8g\" (UID: \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\") " pod="openshift-console/console-59645bbb45-msj8g"
Sep 17 16:44:38.216813 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:38.216619    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cvzg\" (UniqueName: \"kubernetes.io/projected/cd1becfb-1d2f-4670-8f53-516caebc4fb4-kube-api-access-4cvzg\") pod \"console-59645bbb45-msj8g\" (UID: \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\") " pod="openshift-console/console-59645bbb45-msj8g"
Sep 17 16:44:38.216813 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:38.216637    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd1becfb-1d2f-4670-8f53-516caebc4fb4-service-ca\") pod \"console-59645bbb45-msj8g\" (UID: \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\") " pod="openshift-console/console-59645bbb45-msj8g"
Sep 17 16:44:38.216813 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:38.216720    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd1becfb-1d2f-4670-8f53-516caebc4fb4-console-config\") pod \"console-59645bbb45-msj8g\" (UID: \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\") " pod="openshift-console/console-59645bbb45-msj8g"
Sep 17 16:44:38.216813 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:38.216787    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd1becfb-1d2f-4670-8f53-516caebc4fb4-console-serving-cert\") pod \"console-59645bbb45-msj8g\" (UID: \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\") " pod="openshift-console/console-59645bbb45-msj8g"
Sep 17 16:44:38.318025 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:38.317998    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd1becfb-1d2f-4670-8f53-516caebc4fb4-console-config\") pod \"console-59645bbb45-msj8g\" (UID: \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\") " pod="openshift-console/console-59645bbb45-msj8g"
Sep 17 16:44:38.318133 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:38.318039    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd1becfb-1d2f-4670-8f53-516caebc4fb4-console-serving-cert\") pod \"console-59645bbb45-msj8g\" (UID: \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\") " pod="openshift-console/console-59645bbb45-msj8g"
Sep 17 16:44:38.318133 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:38.318074    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd1becfb-1d2f-4670-8f53-516caebc4fb4-trusted-ca-bundle\") pod \"console-59645bbb45-msj8g\" (UID: \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\") " pod="openshift-console/console-59645bbb45-msj8g"
Sep 17 16:44:38.318133 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:38.318091    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd1becfb-1d2f-4670-8f53-516caebc4fb4-oauth-serving-cert\") pod \"console-59645bbb45-msj8g\" (UID: \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\") " pod="openshift-console/console-59645bbb45-msj8g"
Sep 17 16:44:38.318284 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:38.318262    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd1becfb-1d2f-4670-8f53-516caebc4fb4-console-oauth-config\") pod \"console-59645bbb45-msj8g\" (UID: \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\") " pod="openshift-console/console-59645bbb45-msj8g"
Sep 17 16:44:38.318346 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:38.318298    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cvzg\" (UniqueName: \"kubernetes.io/projected/cd1becfb-1d2f-4670-8f53-516caebc4fb4-kube-api-access-4cvzg\") pod \"console-59645bbb45-msj8g\" (UID: \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\") " pod="openshift-console/console-59645bbb45-msj8g"
Sep 17 16:44:38.318346 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:38.318323    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd1becfb-1d2f-4670-8f53-516caebc4fb4-service-ca\") pod \"console-59645bbb45-msj8g\" (UID: \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\") " pod="openshift-console/console-59645bbb45-msj8g"
Sep 17 16:44:38.318856 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:38.318794    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd1becfb-1d2f-4670-8f53-516caebc4fb4-oauth-serving-cert\") pod \"console-59645bbb45-msj8g\" (UID: \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\") " pod="openshift-console/console-59645bbb45-msj8g"
Sep 17 16:44:38.319360 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:38.319336    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd1becfb-1d2f-4670-8f53-516caebc4fb4-console-config\") pod \"console-59645bbb45-msj8g\" (UID: \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\") " pod="openshift-console/console-59645bbb45-msj8g"
Sep 17 16:44:38.319442 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:38.319367    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd1becfb-1d2f-4670-8f53-516caebc4fb4-trusted-ca-bundle\") pod \"console-59645bbb45-msj8g\" (UID: \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\") " pod="openshift-console/console-59645bbb45-msj8g"
Sep 17 16:44:38.319442 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:38.319341    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd1becfb-1d2f-4670-8f53-516caebc4fb4-service-ca\") pod \"console-59645bbb45-msj8g\" (UID: \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\") " pod="openshift-console/console-59645bbb45-msj8g"
Sep 17 16:44:38.320729 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:38.320708    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd1becfb-1d2f-4670-8f53-516caebc4fb4-console-serving-cert\") pod \"console-59645bbb45-msj8g\" (UID: \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\") " pod="openshift-console/console-59645bbb45-msj8g"
Sep 17 16:44:38.321139 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:38.321120    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd1becfb-1d2f-4670-8f53-516caebc4fb4-console-oauth-config\") pod \"console-59645bbb45-msj8g\" (UID: \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\") " pod="openshift-console/console-59645bbb45-msj8g"
Sep 17 16:44:38.327230 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:38.327213    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cvzg\" (UniqueName: \"kubernetes.io/projected/cd1becfb-1d2f-4670-8f53-516caebc4fb4-kube-api-access-4cvzg\") pod \"console-59645bbb45-msj8g\" (UID: \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\") " pod="openshift-console/console-59645bbb45-msj8g"
Sep 17 16:44:38.424613 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:38.424586    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59645bbb45-msj8g"
Sep 17 16:44:38.699009 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:38.698931    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59645bbb45-msj8g"]
Sep 17 16:44:38.702867 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:44:38.702833    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd1becfb_1d2f_4670_8f53_516caebc4fb4.slice/crio-f2f588a0979b3dbb85059832f57bcb29b3a3f2bc690d80026bacd8751bfe9a81 WatchSource:0}: Error finding container f2f588a0979b3dbb85059832f57bcb29b3a3f2bc690d80026bacd8751bfe9a81: Status 404 returned error can't find the container with id f2f588a0979b3dbb85059832f57bcb29b3a3f2bc690d80026bacd8751bfe9a81
Sep 17 16:44:39.013579 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:39.013485    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59645bbb45-msj8g" event={"ID":"cd1becfb-1d2f-4670-8f53-516caebc4fb4","Type":"ContainerStarted","Data":"5360771c114722ad9b64d64cd288edcf5bc7c84c8ec1ebea844e718f283ab251"}
Sep 17 16:44:39.013579 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:39.013547    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59645bbb45-msj8g" event={"ID":"cd1becfb-1d2f-4670-8f53-516caebc4fb4","Type":"ContainerStarted","Data":"f2f588a0979b3dbb85059832f57bcb29b3a3f2bc690d80026bacd8751bfe9a81"}
Sep 17 16:44:39.032421 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:39.032368    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-59645bbb45-msj8g" podStartSLOduration=1.032352388 podStartE2EDuration="1.032352388s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:44:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:44:39.030221477 +0000 UTC m=+149.731300151" watchObservedRunningTime="2026-09-17 16:44:39.032352388 +0000 UTC m=+149.733431047"
Sep 17 16:44:48.425138 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:48.425093    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-59645bbb45-msj8g"
Sep 17 16:44:48.425523 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:48.425161    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59645bbb45-msj8g"
Sep 17 16:44:48.429788 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:48.429764    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-59645bbb45-msj8g"
Sep 17 16:44:49.053992 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:49.053964    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-59645bbb45-msj8g"
Sep 17 16:44:49.102508 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:44:49.102479    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f87766945-8gkhv"]
Sep 17 16:45:14.122576 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:14.122537    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7f87766945-8gkhv" podUID="04213207-3bca-4a89-882a-1cda4ae924f1" containerName="console" containerID="cri-o://7765a4d104608b6ade605e8d25879154cc41687bcb90e5e5cae287a9fc97d0d2" gracePeriod=15
Sep 17 16:45:14.365107 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:14.365084    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f87766945-8gkhv"
Sep 17 16:45:14.391928 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:14.391851    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/04213207-3bca-4a89-882a-1cda4ae924f1-console-oauth-config\") pod \"04213207-3bca-4a89-882a-1cda4ae924f1\" (UID: \"04213207-3bca-4a89-882a-1cda4ae924f1\") "
Sep 17 16:45:14.392050 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:14.391933    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/04213207-3bca-4a89-882a-1cda4ae924f1-service-ca\") pod \"04213207-3bca-4a89-882a-1cda4ae924f1\" (UID: \"04213207-3bca-4a89-882a-1cda4ae924f1\") "
Sep 17 16:45:14.392050 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:14.391952    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04213207-3bca-4a89-882a-1cda4ae924f1-trusted-ca-bundle\") pod \"04213207-3bca-4a89-882a-1cda4ae924f1\" (UID: \"04213207-3bca-4a89-882a-1cda4ae924f1\") "
Sep 17 16:45:14.392050 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:14.391978    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsq54\" (UniqueName: \"kubernetes.io/projected/04213207-3bca-4a89-882a-1cda4ae924f1-kube-api-access-rsq54\") pod \"04213207-3bca-4a89-882a-1cda4ae924f1\" (UID: \"04213207-3bca-4a89-882a-1cda4ae924f1\") "
Sep 17 16:45:14.392050 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:14.392015    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/04213207-3bca-4a89-882a-1cda4ae924f1-console-serving-cert\") pod \"04213207-3bca-4a89-882a-1cda4ae924f1\" (UID: \"04213207-3bca-4a89-882a-1cda4ae924f1\") "
Sep 17 16:45:14.392050 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:14.392037    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/04213207-3bca-4a89-882a-1cda4ae924f1-oauth-serving-cert\") pod \"04213207-3bca-4a89-882a-1cda4ae924f1\" (UID: \"04213207-3bca-4a89-882a-1cda4ae924f1\") "
Sep 17 16:45:14.392298 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:14.392190    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/04213207-3bca-4a89-882a-1cda4ae924f1-console-config\") pod \"04213207-3bca-4a89-882a-1cda4ae924f1\" (UID: \"04213207-3bca-4a89-882a-1cda4ae924f1\") "
Sep 17 16:45:14.392435 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:14.392406    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04213207-3bca-4a89-882a-1cda4ae924f1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "04213207-3bca-4a89-882a-1cda4ae924f1" (UID: "04213207-3bca-4a89-882a-1cda4ae924f1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:45:14.392516 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:14.392441    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04213207-3bca-4a89-882a-1cda4ae924f1-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "04213207-3bca-4a89-882a-1cda4ae924f1" (UID: "04213207-3bca-4a89-882a-1cda4ae924f1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:45:14.392681 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:14.392659    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04213207-3bca-4a89-882a-1cda4ae924f1-service-ca" (OuterVolumeSpecName: "service-ca") pod "04213207-3bca-4a89-882a-1cda4ae924f1" (UID: "04213207-3bca-4a89-882a-1cda4ae924f1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:45:14.393017 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:14.392767    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04213207-3bca-4a89-882a-1cda4ae924f1-console-config" (OuterVolumeSpecName: "console-config") pod "04213207-3bca-4a89-882a-1cda4ae924f1" (UID: "04213207-3bca-4a89-882a-1cda4ae924f1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:45:14.394180 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:14.394155    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04213207-3bca-4a89-882a-1cda4ae924f1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "04213207-3bca-4a89-882a-1cda4ae924f1" (UID: "04213207-3bca-4a89-882a-1cda4ae924f1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:45:14.394317 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:14.394300    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04213207-3bca-4a89-882a-1cda4ae924f1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "04213207-3bca-4a89-882a-1cda4ae924f1" (UID: "04213207-3bca-4a89-882a-1cda4ae924f1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:45:14.394418 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:14.394400    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04213207-3bca-4a89-882a-1cda4ae924f1-kube-api-access-rsq54" (OuterVolumeSpecName: "kube-api-access-rsq54") pod "04213207-3bca-4a89-882a-1cda4ae924f1" (UID: "04213207-3bca-4a89-882a-1cda4ae924f1"). InnerVolumeSpecName "kube-api-access-rsq54". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:45:14.493034 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:14.492991    2586 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/04213207-3bca-4a89-882a-1cda4ae924f1-console-serving-cert\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:45:14.493034 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:14.493029    2586 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/04213207-3bca-4a89-882a-1cda4ae924f1-oauth-serving-cert\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:45:14.493034 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:14.493039    2586 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/04213207-3bca-4a89-882a-1cda4ae924f1-console-config\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:45:14.493034 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:14.493047    2586 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/04213207-3bca-4a89-882a-1cda4ae924f1-console-oauth-config\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:45:14.493281 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:14.493055    2586 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/04213207-3bca-4a89-882a-1cda4ae924f1-service-ca\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:45:14.493281 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:14.493065    2586 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04213207-3bca-4a89-882a-1cda4ae924f1-trusted-ca-bundle\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:45:14.493281 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:14.493072    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsq54\" (UniqueName: \"kubernetes.io/projected/04213207-3bca-4a89-882a-1cda4ae924f1-kube-api-access-rsq54\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:45:15.146823 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:15.146785    2586 generic.go:356] "Generic (PLEG): container finished" podID="04213207-3bca-4a89-882a-1cda4ae924f1" containerID="7765a4d104608b6ade605e8d25879154cc41687bcb90e5e5cae287a9fc97d0d2" exitCode=0
Sep 17 16:45:15.147221 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:15.146833    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f87766945-8gkhv"
Sep 17 16:45:15.147221 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:15.146868    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f87766945-8gkhv" event={"ID":"04213207-3bca-4a89-882a-1cda4ae924f1","Type":"ContainerDied","Data":"7765a4d104608b6ade605e8d25879154cc41687bcb90e5e5cae287a9fc97d0d2"}
Sep 17 16:45:15.147221 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:15.146947    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f87766945-8gkhv" event={"ID":"04213207-3bca-4a89-882a-1cda4ae924f1","Type":"ContainerDied","Data":"7f6e38ff9e5ea3e43054e3dae4e2d3f837177900f90e0be747bc37ac6155d9d8"}
Sep 17 16:45:15.147221 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:15.146968    2586 scope.go:117] "RemoveContainer" containerID="7765a4d104608b6ade605e8d25879154cc41687bcb90e5e5cae287a9fc97d0d2"
Sep 17 16:45:15.155770 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:15.155747    2586 scope.go:117] "RemoveContainer" containerID="7765a4d104608b6ade605e8d25879154cc41687bcb90e5e5cae287a9fc97d0d2"
Sep 17 16:45:15.156162 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:45:15.156144    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7765a4d104608b6ade605e8d25879154cc41687bcb90e5e5cae287a9fc97d0d2\": container with ID starting with 7765a4d104608b6ade605e8d25879154cc41687bcb90e5e5cae287a9fc97d0d2 not found: ID does not exist" containerID="7765a4d104608b6ade605e8d25879154cc41687bcb90e5e5cae287a9fc97d0d2"
Sep 17 16:45:15.156233 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:15.156170    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7765a4d104608b6ade605e8d25879154cc41687bcb90e5e5cae287a9fc97d0d2"} err="failed to get container status \"7765a4d104608b6ade605e8d25879154cc41687bcb90e5e5cae287a9fc97d0d2\": rpc error: code = NotFound desc = could not find container \"7765a4d104608b6ade605e8d25879154cc41687bcb90e5e5cae287a9fc97d0d2\": container with ID starting with 7765a4d104608b6ade605e8d25879154cc41687bcb90e5e5cae287a9fc97d0d2 not found: ID does not exist"
Sep 17 16:45:15.166812 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:15.166783    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f87766945-8gkhv"]
Sep 17 16:45:15.170249 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:15.170225    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f87766945-8gkhv"]
Sep 17 16:45:15.328094 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:15.328059    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04213207-3bca-4a89-882a-1cda4ae924f1" path="/var/lib/kubelet/pods/04213207-3bca-4a89-882a-1cda4ae924f1/volumes"
Sep 17 16:45:52.368726 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:52.368693    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6cc6456699-sfwst"]
Sep 17 16:45:52.369183 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:52.368984    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04213207-3bca-4a89-882a-1cda4ae924f1" containerName="console"
Sep 17 16:45:52.369183 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:52.368996    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="04213207-3bca-4a89-882a-1cda4ae924f1" containerName="console"
Sep 17 16:45:52.369183 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:52.369061    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="04213207-3bca-4a89-882a-1cda4ae924f1" containerName="console"
Sep 17 16:45:52.371699 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:52.371681    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cc6456699-sfwst"
Sep 17 16:45:52.381804 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:52.381777    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cc6456699-sfwst"]
Sep 17 16:45:52.481447 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:52.481405    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc010cbe-909b-4b0a-b293-92627cb28412-console-config\") pod \"console-6cc6456699-sfwst\" (UID: \"fc010cbe-909b-4b0a-b293-92627cb28412\") " pod="openshift-console/console-6cc6456699-sfwst"
Sep 17 16:45:52.481626 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:52.481458    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc010cbe-909b-4b0a-b293-92627cb28412-console-serving-cert\") pod \"console-6cc6456699-sfwst\" (UID: \"fc010cbe-909b-4b0a-b293-92627cb28412\") " pod="openshift-console/console-6cc6456699-sfwst"
Sep 17 16:45:52.481626 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:52.481497    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc010cbe-909b-4b0a-b293-92627cb28412-trusted-ca-bundle\") pod \"console-6cc6456699-sfwst\" (UID: \"fc010cbe-909b-4b0a-b293-92627cb28412\") " pod="openshift-console/console-6cc6456699-sfwst"
Sep 17 16:45:52.481626 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:52.481513    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h682t\" (UniqueName: \"kubernetes.io/projected/fc010cbe-909b-4b0a-b293-92627cb28412-kube-api-access-h682t\") pod \"console-6cc6456699-sfwst\" (UID: \"fc010cbe-909b-4b0a-b293-92627cb28412\") " pod="openshift-console/console-6cc6456699-sfwst"
Sep 17 16:45:52.481626 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:52.481545    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc010cbe-909b-4b0a-b293-92627cb28412-oauth-serving-cert\") pod \"console-6cc6456699-sfwst\" (UID: \"fc010cbe-909b-4b0a-b293-92627cb28412\") " pod="openshift-console/console-6cc6456699-sfwst"
Sep 17 16:45:52.481626 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:52.481572    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc010cbe-909b-4b0a-b293-92627cb28412-console-oauth-config\") pod \"console-6cc6456699-sfwst\" (UID: \"fc010cbe-909b-4b0a-b293-92627cb28412\") " pod="openshift-console/console-6cc6456699-sfwst"
Sep 17 16:45:52.481786 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:52.481647    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc010cbe-909b-4b0a-b293-92627cb28412-service-ca\") pod \"console-6cc6456699-sfwst\" (UID: \"fc010cbe-909b-4b0a-b293-92627cb28412\") " pod="openshift-console/console-6cc6456699-sfwst"
Sep 17 16:45:52.582591 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:52.582553    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc010cbe-909b-4b0a-b293-92627cb28412-console-serving-cert\") pod \"console-6cc6456699-sfwst\" (UID: \"fc010cbe-909b-4b0a-b293-92627cb28412\") " pod="openshift-console/console-6cc6456699-sfwst"
Sep 17 16:45:52.582591 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:52.582596    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc010cbe-909b-4b0a-b293-92627cb28412-trusted-ca-bundle\") pod \"console-6cc6456699-sfwst\" (UID: \"fc010cbe-909b-4b0a-b293-92627cb28412\") " pod="openshift-console/console-6cc6456699-sfwst"
Sep 17 16:45:52.582835 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:52.582616    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h682t\" (UniqueName: \"kubernetes.io/projected/fc010cbe-909b-4b0a-b293-92627cb28412-kube-api-access-h682t\") pod \"console-6cc6456699-sfwst\" (UID: \"fc010cbe-909b-4b0a-b293-92627cb28412\") " pod="openshift-console/console-6cc6456699-sfwst"
Sep 17 16:45:52.582835 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:52.582670    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc010cbe-909b-4b0a-b293-92627cb28412-oauth-serving-cert\") pod \"console-6cc6456699-sfwst\" (UID: \"fc010cbe-909b-4b0a-b293-92627cb28412\") " pod="openshift-console/console-6cc6456699-sfwst"
Sep 17 16:45:52.582835 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:52.582696    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc010cbe-909b-4b0a-b293-92627cb28412-console-oauth-config\") pod \"console-6cc6456699-sfwst\" (UID: \"fc010cbe-909b-4b0a-b293-92627cb28412\") " pod="openshift-console/console-6cc6456699-sfwst"
Sep 17 16:45:52.582835 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:52.582738    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc010cbe-909b-4b0a-b293-92627cb28412-service-ca\") pod \"console-6cc6456699-sfwst\" (UID: \"fc010cbe-909b-4b0a-b293-92627cb28412\") " pod="openshift-console/console-6cc6456699-sfwst"
Sep 17 16:45:52.582835 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:52.582764    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc010cbe-909b-4b0a-b293-92627cb28412-console-config\") pod \"console-6cc6456699-sfwst\" (UID: \"fc010cbe-909b-4b0a-b293-92627cb28412\") " pod="openshift-console/console-6cc6456699-sfwst"
Sep 17 16:45:52.583571 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:52.583538    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc010cbe-909b-4b0a-b293-92627cb28412-console-config\") pod \"console-6cc6456699-sfwst\" (UID: \"fc010cbe-909b-4b0a-b293-92627cb28412\") " pod="openshift-console/console-6cc6456699-sfwst"
Sep 17 16:45:52.583694 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:52.583572    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc010cbe-909b-4b0a-b293-92627cb28412-service-ca\") pod \"console-6cc6456699-sfwst\" (UID: \"fc010cbe-909b-4b0a-b293-92627cb28412\") " pod="openshift-console/console-6cc6456699-sfwst"
Sep 17 16:45:52.583694 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:52.583600    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc010cbe-909b-4b0a-b293-92627cb28412-trusted-ca-bundle\") pod \"console-6cc6456699-sfwst\" (UID: \"fc010cbe-909b-4b0a-b293-92627cb28412\") " pod="openshift-console/console-6cc6456699-sfwst"
Sep 17 16:45:52.583806 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:52.583781    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc010cbe-909b-4b0a-b293-92627cb28412-oauth-serving-cert\") pod \"console-6cc6456699-sfwst\" (UID: \"fc010cbe-909b-4b0a-b293-92627cb28412\") " pod="openshift-console/console-6cc6456699-sfwst"
Sep 17 16:45:52.585314 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:52.585292    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc010cbe-909b-4b0a-b293-92627cb28412-console-serving-cert\") pod \"console-6cc6456699-sfwst\" (UID: \"fc010cbe-909b-4b0a-b293-92627cb28412\") " pod="openshift-console/console-6cc6456699-sfwst"
Sep 17 16:45:52.585417 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:52.585357    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc010cbe-909b-4b0a-b293-92627cb28412-console-oauth-config\") pod \"console-6cc6456699-sfwst\" (UID: \"fc010cbe-909b-4b0a-b293-92627cb28412\") " pod="openshift-console/console-6cc6456699-sfwst"
Sep 17 16:45:52.590952 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:52.590931    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h682t\" (UniqueName: \"kubernetes.io/projected/fc010cbe-909b-4b0a-b293-92627cb28412-kube-api-access-h682t\") pod \"console-6cc6456699-sfwst\" (UID: \"fc010cbe-909b-4b0a-b293-92627cb28412\") " pod="openshift-console/console-6cc6456699-sfwst"
Sep 17 16:45:52.681140 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:52.681044    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cc6456699-sfwst"
Sep 17 16:45:52.952592 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:52.952568    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cc6456699-sfwst"]
Sep 17 16:45:52.955257 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:45:52.955226    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc010cbe_909b_4b0a_b293_92627cb28412.slice/crio-91eb9d61dc734adcc57086da5958d072c2a859d963c495550e7b7c27908af229 WatchSource:0}: Error finding container 91eb9d61dc734adcc57086da5958d072c2a859d963c495550e7b7c27908af229: Status 404 returned error can't find the container with id 91eb9d61dc734adcc57086da5958d072c2a859d963c495550e7b7c27908af229
Sep 17 16:45:53.280708 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:53.280670    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cc6456699-sfwst" event={"ID":"fc010cbe-909b-4b0a-b293-92627cb28412","Type":"ContainerStarted","Data":"ae4eb31bd436d24b8bdd9f7f21d34e11ddfdc5e184d74180902879df0e8a4bc4"}
Sep 17 16:45:53.280922 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:53.280719    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cc6456699-sfwst" event={"ID":"fc010cbe-909b-4b0a-b293-92627cb28412","Type":"ContainerStarted","Data":"91eb9d61dc734adcc57086da5958d072c2a859d963c495550e7b7c27908af229"}
Sep 17 16:45:53.298787 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:45:53.298729    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6cc6456699-sfwst" podStartSLOduration=1.2987112459999999 podStartE2EDuration="1.298711246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:45:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:45:53.298272616 +0000 UTC m=+223.999351290" watchObservedRunningTime="2026-09-17 16:45:53.298711246 +0000 UTC m=+223.999789965"
Sep 17 16:46:02.681165 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:02.681129    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6cc6456699-sfwst"
Sep 17 16:46:02.681590 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:02.681222    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6cc6456699-sfwst"
Sep 17 16:46:02.686022 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:02.685999    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6cc6456699-sfwst"
Sep 17 16:46:03.323715 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:03.323684    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6cc6456699-sfwst"
Sep 17 16:46:03.369930 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:03.369900    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59645bbb45-msj8g"]
Sep 17 16:46:28.389562 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:28.389452    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-59645bbb45-msj8g" podUID="cd1becfb-1d2f-4670-8f53-516caebc4fb4" containerName="console" containerID="cri-o://5360771c114722ad9b64d64cd288edcf5bc7c84c8ec1ebea844e718f283ab251" gracePeriod=15
Sep 17 16:46:28.631942 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:28.631917    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59645bbb45-msj8g"
Sep 17 16:46:28.679266 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:28.679174    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd1becfb-1d2f-4670-8f53-516caebc4fb4-trusted-ca-bundle\") pod \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\" (UID: \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\") "
Sep 17 16:46:28.679266 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:28.679215    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4cvzg\" (UniqueName: \"kubernetes.io/projected/cd1becfb-1d2f-4670-8f53-516caebc4fb4-kube-api-access-4cvzg\") pod \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\" (UID: \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\") "
Sep 17 16:46:28.679266 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:28.679232    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd1becfb-1d2f-4670-8f53-516caebc4fb4-console-serving-cert\") pod \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\" (UID: \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\") "
Sep 17 16:46:28.679266 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:28.679257    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd1becfb-1d2f-4670-8f53-516caebc4fb4-service-ca\") pod \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\" (UID: \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\") "
Sep 17 16:46:28.679596 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:28.679314    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd1becfb-1d2f-4670-8f53-516caebc4fb4-console-oauth-config\") pod \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\" (UID: \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\") "
Sep 17 16:46:28.679596 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:28.679343    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd1becfb-1d2f-4670-8f53-516caebc4fb4-oauth-serving-cert\") pod \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\" (UID: \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\") "
Sep 17 16:46:28.679596 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:28.679380    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd1becfb-1d2f-4670-8f53-516caebc4fb4-console-config\") pod \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\" (UID: \"cd1becfb-1d2f-4670-8f53-516caebc4fb4\") "
Sep 17 16:46:28.679736 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:28.679637    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd1becfb-1d2f-4670-8f53-516caebc4fb4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "cd1becfb-1d2f-4670-8f53-516caebc4fb4" (UID: "cd1becfb-1d2f-4670-8f53-516caebc4fb4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:46:28.679837 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:28.679813    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd1becfb-1d2f-4670-8f53-516caebc4fb4-console-config" (OuterVolumeSpecName: "console-config") pod "cd1becfb-1d2f-4670-8f53-516caebc4fb4" (UID: "cd1becfb-1d2f-4670-8f53-516caebc4fb4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:46:28.679837 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:28.679810    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd1becfb-1d2f-4670-8f53-516caebc4fb4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cd1becfb-1d2f-4670-8f53-516caebc4fb4" (UID: "cd1becfb-1d2f-4670-8f53-516caebc4fb4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:46:28.680006 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:28.679854    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd1becfb-1d2f-4670-8f53-516caebc4fb4-service-ca" (OuterVolumeSpecName: "service-ca") pod "cd1becfb-1d2f-4670-8f53-516caebc4fb4" (UID: "cd1becfb-1d2f-4670-8f53-516caebc4fb4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:46:28.681619 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:28.681595    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd1becfb-1d2f-4670-8f53-516caebc4fb4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cd1becfb-1d2f-4670-8f53-516caebc4fb4" (UID: "cd1becfb-1d2f-4670-8f53-516caebc4fb4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:46:28.681619 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:28.681602    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd1becfb-1d2f-4670-8f53-516caebc4fb4-kube-api-access-4cvzg" (OuterVolumeSpecName: "kube-api-access-4cvzg") pod "cd1becfb-1d2f-4670-8f53-516caebc4fb4" (UID: "cd1becfb-1d2f-4670-8f53-516caebc4fb4"). InnerVolumeSpecName "kube-api-access-4cvzg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:46:28.681804 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:28.681785    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd1becfb-1d2f-4670-8f53-516caebc4fb4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cd1becfb-1d2f-4670-8f53-516caebc4fb4" (UID: "cd1becfb-1d2f-4670-8f53-516caebc4fb4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:46:28.780545 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:28.780502    2586 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd1becfb-1d2f-4670-8f53-516caebc4fb4-service-ca\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:46:28.780545 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:28.780537    2586 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd1becfb-1d2f-4670-8f53-516caebc4fb4-console-oauth-config\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:46:28.780545 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:28.780549    2586 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd1becfb-1d2f-4670-8f53-516caebc4fb4-oauth-serving-cert\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:46:28.780796 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:28.780561    2586 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd1becfb-1d2f-4670-8f53-516caebc4fb4-console-config\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:46:28.780796 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:28.780574    2586 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd1becfb-1d2f-4670-8f53-516caebc4fb4-trusted-ca-bundle\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:46:28.780796 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:28.780585    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cvzg\" (UniqueName: \"kubernetes.io/projected/cd1becfb-1d2f-4670-8f53-516caebc4fb4-kube-api-access-4cvzg\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:46:28.780796 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:28.780596    2586 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd1becfb-1d2f-4670-8f53-516caebc4fb4-console-serving-cert\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:46:29.416826 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:29.416786    2586 generic.go:356] "Generic (PLEG): container finished" podID="cd1becfb-1d2f-4670-8f53-516caebc4fb4" containerID="5360771c114722ad9b64d64cd288edcf5bc7c84c8ec1ebea844e718f283ab251" exitCode=0
Sep 17 16:46:29.417267 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:29.416846    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59645bbb45-msj8g"
Sep 17 16:46:29.417267 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:29.416865    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59645bbb45-msj8g" event={"ID":"cd1becfb-1d2f-4670-8f53-516caebc4fb4","Type":"ContainerDied","Data":"5360771c114722ad9b64d64cd288edcf5bc7c84c8ec1ebea844e718f283ab251"}
Sep 17 16:46:29.417267 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:29.416956    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59645bbb45-msj8g" event={"ID":"cd1becfb-1d2f-4670-8f53-516caebc4fb4","Type":"ContainerDied","Data":"f2f588a0979b3dbb85059832f57bcb29b3a3f2bc690d80026bacd8751bfe9a81"}
Sep 17 16:46:29.417267 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:29.416977    2586 scope.go:117] "RemoveContainer" containerID="5360771c114722ad9b64d64cd288edcf5bc7c84c8ec1ebea844e718f283ab251"
Sep 17 16:46:29.425186 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:29.425171    2586 scope.go:117] "RemoveContainer" containerID="5360771c114722ad9b64d64cd288edcf5bc7c84c8ec1ebea844e718f283ab251"
Sep 17 16:46:29.425447 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:46:29.425429    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5360771c114722ad9b64d64cd288edcf5bc7c84c8ec1ebea844e718f283ab251\": container with ID starting with 5360771c114722ad9b64d64cd288edcf5bc7c84c8ec1ebea844e718f283ab251 not found: ID does not exist" containerID="5360771c114722ad9b64d64cd288edcf5bc7c84c8ec1ebea844e718f283ab251"
Sep 17 16:46:29.425500 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:29.425455    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5360771c114722ad9b64d64cd288edcf5bc7c84c8ec1ebea844e718f283ab251"} err="failed to get container status \"5360771c114722ad9b64d64cd288edcf5bc7c84c8ec1ebea844e718f283ab251\": rpc error: code = NotFound desc = could not find container \"5360771c114722ad9b64d64cd288edcf5bc7c84c8ec1ebea844e718f283ab251\": container with ID starting with 5360771c114722ad9b64d64cd288edcf5bc7c84c8ec1ebea844e718f283ab251 not found: ID does not exist"
Sep 17 16:46:29.433684 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:29.433655    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59645bbb45-msj8g"]
Sep 17 16:46:29.439032 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:29.439010    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-59645bbb45-msj8g"]
Sep 17 16:46:31.324860 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:31.324825    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd1becfb-1d2f-4670-8f53-516caebc4fb4" path="/var/lib/kubelet/pods/cd1becfb-1d2f-4670-8f53-516caebc4fb4/volumes"
Sep 17 16:46:54.745685 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:54.745647    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2dxlx"]
Sep 17 16:46:54.746121 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:54.745944    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd1becfb-1d2f-4670-8f53-516caebc4fb4" containerName="console"
Sep 17 16:46:54.746121 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:54.745957    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd1becfb-1d2f-4670-8f53-516caebc4fb4" containerName="console"
Sep 17 16:46:54.746121 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:54.746014    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd1becfb-1d2f-4670-8f53-516caebc4fb4" containerName="console"
Sep 17 16:46:54.748913 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:54.748896    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2dxlx"
Sep 17 16:46:54.751810 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:54.751790    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-hcg6t\""
Sep 17 16:46:54.752029 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:54.752015    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 17 16:46:54.752809 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:54.752795    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 17 16:46:54.757359 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:54.757334    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2dxlx"]
Sep 17 16:46:54.877474 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:54.877432    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5a23dd40-5605-48c8-baae-90bc1eaa22a7-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2dxlx\" (UID: \"5a23dd40-5605-48c8-baae-90bc1eaa22a7\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2dxlx"
Sep 17 16:46:54.877474 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:54.877480    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22fgc\" (UniqueName: \"kubernetes.io/projected/5a23dd40-5605-48c8-baae-90bc1eaa22a7-kube-api-access-22fgc\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2dxlx\" (UID: \"5a23dd40-5605-48c8-baae-90bc1eaa22a7\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2dxlx"
Sep 17 16:46:54.877733 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:54.877586    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5a23dd40-5605-48c8-baae-90bc1eaa22a7-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2dxlx\" (UID: \"5a23dd40-5605-48c8-baae-90bc1eaa22a7\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2dxlx"
Sep 17 16:46:54.978980 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:54.978947    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5a23dd40-5605-48c8-baae-90bc1eaa22a7-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2dxlx\" (UID: \"5a23dd40-5605-48c8-baae-90bc1eaa22a7\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2dxlx"
Sep 17 16:46:54.979146 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:54.979000    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5a23dd40-5605-48c8-baae-90bc1eaa22a7-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2dxlx\" (UID: \"5a23dd40-5605-48c8-baae-90bc1eaa22a7\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2dxlx"
Sep 17 16:46:54.979146 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:54.979040    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22fgc\" (UniqueName: \"kubernetes.io/projected/5a23dd40-5605-48c8-baae-90bc1eaa22a7-kube-api-access-22fgc\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2dxlx\" (UID: \"5a23dd40-5605-48c8-baae-90bc1eaa22a7\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2dxlx"
Sep 17 16:46:54.979423 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:54.979399    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5a23dd40-5605-48c8-baae-90bc1eaa22a7-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2dxlx\" (UID: \"5a23dd40-5605-48c8-baae-90bc1eaa22a7\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2dxlx"
Sep 17 16:46:54.979465 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:54.979407    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5a23dd40-5605-48c8-baae-90bc1eaa22a7-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2dxlx\" (UID: \"5a23dd40-5605-48c8-baae-90bc1eaa22a7\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2dxlx"
Sep 17 16:46:54.988169 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:54.988141    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22fgc\" (UniqueName: \"kubernetes.io/projected/5a23dd40-5605-48c8-baae-90bc1eaa22a7-kube-api-access-22fgc\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2dxlx\" (UID: \"5a23dd40-5605-48c8-baae-90bc1eaa22a7\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2dxlx"
Sep 17 16:46:55.058141 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:55.058103    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2dxlx"
Sep 17 16:46:55.328711 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:55.328617    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2dxlx"]
Sep 17 16:46:55.332576 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:46:55.332553    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a23dd40_5605_48c8_baae_90bc1eaa22a7.slice/crio-c2774765c98af2ecbdd5823ad140b935ec03a59affc98d747b863f40d4cb627d WatchSource:0}: Error finding container c2774765c98af2ecbdd5823ad140b935ec03a59affc98d747b863f40d4cb627d: Status 404 returned error can't find the container with id c2774765c98af2ecbdd5823ad140b935ec03a59affc98d747b863f40d4cb627d
Sep 17 16:46:55.516344 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:46:55.516307    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2dxlx" event={"ID":"5a23dd40-5605-48c8-baae-90bc1eaa22a7","Type":"ContainerStarted","Data":"c2774765c98af2ecbdd5823ad140b935ec03a59affc98d747b863f40d4cb627d"}
Sep 17 16:47:03.549307 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:03.549268    2586 generic.go:356] "Generic (PLEG): container finished" podID="5a23dd40-5605-48c8-baae-90bc1eaa22a7" containerID="9a3d11e6c2283e179eba42fc9d14b9daadcf5c9bcfb40896ac4bf7898031b77b" exitCode=0
Sep 17 16:47:03.549708 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:03.549319    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2dxlx" event={"ID":"5a23dd40-5605-48c8-baae-90bc1eaa22a7","Type":"ContainerDied","Data":"9a3d11e6c2283e179eba42fc9d14b9daadcf5c9bcfb40896ac4bf7898031b77b"}
Sep 17 16:47:06.565839 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:06.565805    2586 generic.go:356] "Generic (PLEG): container finished" podID="5a23dd40-5605-48c8-baae-90bc1eaa22a7" containerID="bc7d59964c7bed6bd87580cea5fcbc94a0770bb0023152e8d9356b9d9b74f1f4" exitCode=0
Sep 17 16:47:06.566242 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:06.565853    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2dxlx" event={"ID":"5a23dd40-5605-48c8-baae-90bc1eaa22a7","Type":"ContainerDied","Data":"bc7d59964c7bed6bd87580cea5fcbc94a0770bb0023152e8d9356b9d9b74f1f4"}
Sep 17 16:47:12.870176 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:12.870149    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22fdn_68559fca-75ed-4ee8-bedd-57b04fbd3f4d/ovn-acl-logging/0.log"
Sep 17 16:47:12.870609 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:12.870153    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22fdn_68559fca-75ed-4ee8-bedd-57b04fbd3f4d/ovn-acl-logging/0.log"
Sep 17 16:47:12.876581 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:12.876556    2586 kubelet.go:1648] "Image garbage collection succeeded"
Sep 17 16:47:13.599013 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:13.598976    2586 generic.go:356] "Generic (PLEG): container finished" podID="5a23dd40-5605-48c8-baae-90bc1eaa22a7" containerID="4178d5503f410f059e140f89eed5ff704dd0c42066aff1843261c53d88639278" exitCode=0
Sep 17 16:47:13.599205 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:13.599028    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2dxlx" event={"ID":"5a23dd40-5605-48c8-baae-90bc1eaa22a7","Type":"ContainerDied","Data":"4178d5503f410f059e140f89eed5ff704dd0c42066aff1843261c53d88639278"}
Sep 17 16:47:14.736921 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:14.736865    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2dxlx"
Sep 17 16:47:14.855289 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:14.855174    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22fgc\" (UniqueName: \"kubernetes.io/projected/5a23dd40-5605-48c8-baae-90bc1eaa22a7-kube-api-access-22fgc\") pod \"5a23dd40-5605-48c8-baae-90bc1eaa22a7\" (UID: \"5a23dd40-5605-48c8-baae-90bc1eaa22a7\") "
Sep 17 16:47:14.855289 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:14.855276    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5a23dd40-5605-48c8-baae-90bc1eaa22a7-bundle\") pod \"5a23dd40-5605-48c8-baae-90bc1eaa22a7\" (UID: \"5a23dd40-5605-48c8-baae-90bc1eaa22a7\") "
Sep 17 16:47:14.855514 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:14.855326    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5a23dd40-5605-48c8-baae-90bc1eaa22a7-util\") pod \"5a23dd40-5605-48c8-baae-90bc1eaa22a7\" (UID: \"5a23dd40-5605-48c8-baae-90bc1eaa22a7\") "
Sep 17 16:47:14.855993 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:14.855957    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a23dd40-5605-48c8-baae-90bc1eaa22a7-bundle" (OuterVolumeSpecName: "bundle") pod "5a23dd40-5605-48c8-baae-90bc1eaa22a7" (UID: "5a23dd40-5605-48c8-baae-90bc1eaa22a7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 16:47:14.857692 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:14.857665    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a23dd40-5605-48c8-baae-90bc1eaa22a7-kube-api-access-22fgc" (OuterVolumeSpecName: "kube-api-access-22fgc") pod "5a23dd40-5605-48c8-baae-90bc1eaa22a7" (UID: "5a23dd40-5605-48c8-baae-90bc1eaa22a7"). InnerVolumeSpecName "kube-api-access-22fgc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:47:14.861280 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:14.861232    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a23dd40-5605-48c8-baae-90bc1eaa22a7-util" (OuterVolumeSpecName: "util") pod "5a23dd40-5605-48c8-baae-90bc1eaa22a7" (UID: "5a23dd40-5605-48c8-baae-90bc1eaa22a7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 16:47:14.956298 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:14.956251    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22fgc\" (UniqueName: \"kubernetes.io/projected/5a23dd40-5605-48c8-baae-90bc1eaa22a7-kube-api-access-22fgc\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:47:14.956298 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:14.956289    2586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5a23dd40-5605-48c8-baae-90bc1eaa22a7-bundle\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:47:14.956298 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:14.956300    2586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5a23dd40-5605-48c8-baae-90bc1eaa22a7-util\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:47:15.609092 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:15.608996    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2dxlx" event={"ID":"5a23dd40-5605-48c8-baae-90bc1eaa22a7","Type":"ContainerDied","Data":"c2774765c98af2ecbdd5823ad140b935ec03a59affc98d747b863f40d4cb627d"}
Sep 17 16:47:15.609092 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:15.609045    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a2dxlx"
Sep 17 16:47:15.609092 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:15.609059    2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c2774765c98af2ecbdd5823ad140b935ec03a59affc98d747b863f40d4cb627d"
Sep 17 16:47:22.657342 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:22.657301    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-lzk5t"]
Sep 17 16:47:22.657735 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:22.657604    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a23dd40-5605-48c8-baae-90bc1eaa22a7" containerName="extract"
Sep 17 16:47:22.657735 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:22.657617    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a23dd40-5605-48c8-baae-90bc1eaa22a7" containerName="extract"
Sep 17 16:47:22.657735 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:22.657634    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a23dd40-5605-48c8-baae-90bc1eaa22a7" containerName="util"
Sep 17 16:47:22.657735 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:22.657641    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a23dd40-5605-48c8-baae-90bc1eaa22a7" containerName="util"
Sep 17 16:47:22.657735 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:22.657648    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a23dd40-5605-48c8-baae-90bc1eaa22a7" containerName="pull"
Sep 17 16:47:22.657735 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:22.657654    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a23dd40-5605-48c8-baae-90bc1eaa22a7" containerName="pull"
Sep 17 16:47:22.657735 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:22.657707    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a23dd40-5605-48c8-baae-90bc1eaa22a7" containerName="extract"
Sep 17 16:47:22.660389 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:22.660371    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-lzk5t"
Sep 17 16:47:22.663236 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:22.663213    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Sep 17 16:47:22.663450 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:22.663435    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Sep 17 16:47:22.663499 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:22.663434    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Sep 17 16:47:22.663499 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:22.663481    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-49x2r\""
Sep 17 16:47:22.676926 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:22.676893    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-lzk5t"]
Sep 17 16:47:22.818944 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:22.818860    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/624a3184-aae8-466b-9b52-306a4771fb9c-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-lzk5t\" (UID: \"624a3184-aae8-466b-9b52-306a4771fb9c\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-lzk5t"
Sep 17 16:47:22.819119 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:22.819011    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmlkx\" (UniqueName: \"kubernetes.io/projected/624a3184-aae8-466b-9b52-306a4771fb9c-kube-api-access-fmlkx\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-lzk5t\" (UID: \"624a3184-aae8-466b-9b52-306a4771fb9c\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-lzk5t"
Sep 17 16:47:22.920451 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:22.920344    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmlkx\" (UniqueName: \"kubernetes.io/projected/624a3184-aae8-466b-9b52-306a4771fb9c-kube-api-access-fmlkx\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-lzk5t\" (UID: \"624a3184-aae8-466b-9b52-306a4771fb9c\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-lzk5t"
Sep 17 16:47:22.920451 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:22.920406    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/624a3184-aae8-466b-9b52-306a4771fb9c-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-lzk5t\" (UID: \"624a3184-aae8-466b-9b52-306a4771fb9c\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-lzk5t"
Sep 17 16:47:22.923038 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:22.923009    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/624a3184-aae8-466b-9b52-306a4771fb9c-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-lzk5t\" (UID: \"624a3184-aae8-466b-9b52-306a4771fb9c\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-lzk5t"
Sep 17 16:47:22.930389 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:22.930362    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmlkx\" (UniqueName: \"kubernetes.io/projected/624a3184-aae8-466b-9b52-306a4771fb9c-kube-api-access-fmlkx\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-lzk5t\" (UID: \"624a3184-aae8-466b-9b52-306a4771fb9c\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-lzk5t"
Sep 17 16:47:22.970241 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:22.970206    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-lzk5t"
Sep 17 16:47:23.249533 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:23.249459    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-lzk5t"]
Sep 17 16:47:23.252664 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:47:23.252637    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod624a3184_aae8_466b_9b52_306a4771fb9c.slice/crio-4eaefffbdd4c8ce4e948e8a0929ed280d852315083d218d7c558a40b46fe4ad0 WatchSource:0}: Error finding container 4eaefffbdd4c8ce4e948e8a0929ed280d852315083d218d7c558a40b46fe4ad0: Status 404 returned error can't find the container with id 4eaefffbdd4c8ce4e948e8a0929ed280d852315083d218d7c558a40b46fe4ad0
Sep 17 16:47:23.254623 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:23.254600    2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 16:47:23.641388 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:23.641353    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-lzk5t" event={"ID":"624a3184-aae8-466b-9b52-306a4771fb9c","Type":"ContainerStarted","Data":"4eaefffbdd4c8ce4e948e8a0929ed280d852315083d218d7c558a40b46fe4ad0"}
Sep 17 16:47:26.653971 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:26.653931    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-lzk5t" event={"ID":"624a3184-aae8-466b-9b52-306a4771fb9c","Type":"ContainerStarted","Data":"7f77868cdacd67cdebf7facffe8b7f21c751f3d2a937ea8637741b500426929c"}
Sep 17 16:47:26.654417 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:26.654078    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-lzk5t"
Sep 17 16:47:26.675754 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:26.675652    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-lzk5t" podStartSLOduration=1.746697228 podStartE2EDuration="4.675634964s" totalImagesPullingTime="2.928937736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:47:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:47:26.673820101 +0000 UTC m=+317.374898766" watchObservedRunningTime="2026-09-17 16:47:26.675634964 +0000 UTC m=+317.376713628"
Sep 17 16:47:26.974778 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:26.974692    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-579c8958f7-hxhdw"]
Sep 17 16:47:26.977919 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:26.977899    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-579c8958f7-hxhdw"
Sep 17 16:47:26.981965 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:26.981935    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Sep 17 16:47:26.982091 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:26.981984    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Sep 17 16:47:26.982146 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:26.982124    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-5hmzt\""
Sep 17 16:47:26.989903 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:26.989852    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-579c8958f7-hxhdw"]
Sep 17 16:47:27.158762 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.158728    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n88jm\" (UniqueName: \"kubernetes.io/projected/1421cd1e-dde5-4eb7-bbbf-4a89f59cb475-kube-api-access-n88jm\") pod \"keda-operator-579c8958f7-hxhdw\" (UID: \"1421cd1e-dde5-4eb7-bbbf-4a89f59cb475\") " pod="openshift-keda/keda-operator-579c8958f7-hxhdw"
Sep 17 16:47:27.158956 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.158831    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/1421cd1e-dde5-4eb7-bbbf-4a89f59cb475-cabundle0\") pod \"keda-operator-579c8958f7-hxhdw\" (UID: \"1421cd1e-dde5-4eb7-bbbf-4a89f59cb475\") " pod="openshift-keda/keda-operator-579c8958f7-hxhdw"
Sep 17 16:47:27.158956 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.158887    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/1421cd1e-dde5-4eb7-bbbf-4a89f59cb475-certificates\") pod \"keda-operator-579c8958f7-hxhdw\" (UID: \"1421cd1e-dde5-4eb7-bbbf-4a89f59cb475\") " pod="openshift-keda/keda-operator-579c8958f7-hxhdw"
Sep 17 16:47:27.260032 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.259930    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/1421cd1e-dde5-4eb7-bbbf-4a89f59cb475-certificates\") pod \"keda-operator-579c8958f7-hxhdw\" (UID: \"1421cd1e-dde5-4eb7-bbbf-4a89f59cb475\") " pod="openshift-keda/keda-operator-579c8958f7-hxhdw"
Sep 17 16:47:27.260032 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.259988    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n88jm\" (UniqueName: \"kubernetes.io/projected/1421cd1e-dde5-4eb7-bbbf-4a89f59cb475-kube-api-access-n88jm\") pod \"keda-operator-579c8958f7-hxhdw\" (UID: \"1421cd1e-dde5-4eb7-bbbf-4a89f59cb475\") " pod="openshift-keda/keda-operator-579c8958f7-hxhdw"
Sep 17 16:47:27.260275 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.260041    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/1421cd1e-dde5-4eb7-bbbf-4a89f59cb475-cabundle0\") pod \"keda-operator-579c8958f7-hxhdw\" (UID: \"1421cd1e-dde5-4eb7-bbbf-4a89f59cb475\") " pod="openshift-keda/keda-operator-579c8958f7-hxhdw"
Sep 17 16:47:27.260275 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:47:27.260113    2586 projected.go:266] Couldn't get secret openshift-keda/keda-operator-certs: secret "keda-operator-certs" not found
Sep 17 16:47:27.260275 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:47:27.260142    2586 secret.go:281] references non-existent secret key: ca.crt
Sep 17 16:47:27.260275 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:47:27.260153    2586 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Sep 17 16:47:27.260275 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:47:27.260168    2586 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-579c8958f7-hxhdw: [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Sep 17 16:47:27.260275 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:47:27.260241    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1421cd1e-dde5-4eb7-bbbf-4a89f59cb475-certificates podName:1421cd1e-dde5-4eb7-bbbf-4a89f59cb475 nodeName:}" failed. No retries permitted until 2026-09-17 16:47:27.760219446 +0000 UTC m=+318.461298088 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/1421cd1e-dde5-4eb7-bbbf-4a89f59cb475-certificates") pod "keda-operator-579c8958f7-hxhdw" (UID: "1421cd1e-dde5-4eb7-bbbf-4a89f59cb475") : [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Sep 17 16:47:27.260634 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.260616    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/1421cd1e-dde5-4eb7-bbbf-4a89f59cb475-cabundle0\") pod \"keda-operator-579c8958f7-hxhdw\" (UID: \"1421cd1e-dde5-4eb7-bbbf-4a89f59cb475\") " pod="openshift-keda/keda-operator-579c8958f7-hxhdw"
Sep 17 16:47:27.272917 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.272893    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n88jm\" (UniqueName: \"kubernetes.io/projected/1421cd1e-dde5-4eb7-bbbf-4a89f59cb475-kube-api-access-n88jm\") pod \"keda-operator-579c8958f7-hxhdw\" (UID: \"1421cd1e-dde5-4eb7-bbbf-4a89f59cb475\") " pod="openshift-keda/keda-operator-579c8958f7-hxhdw"
Sep 17 16:47:27.297848 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.297820    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-crzsn"]
Sep 17 16:47:27.301048 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.301032    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-crzsn"
Sep 17 16:47:27.303431 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.303414    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Sep 17 16:47:27.309747 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.309724    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-crzsn"]
Sep 17 16:47:27.462249 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.462212    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svrq8\" (UniqueName: \"kubernetes.io/projected/424a4a19-21fe-47d0-bfd7-45122285a714-kube-api-access-svrq8\") pod \"keda-metrics-apiserver-77857b7b98-crzsn\" (UID: \"424a4a19-21fe-47d0-bfd7-45122285a714\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-crzsn"
Sep 17 16:47:27.462402 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.462259    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/424a4a19-21fe-47d0-bfd7-45122285a714-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-crzsn\" (UID: \"424a4a19-21fe-47d0-bfd7-45122285a714\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-crzsn"
Sep 17 16:47:27.462468 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.462398    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/424a4a19-21fe-47d0-bfd7-45122285a714-certificates\") pod \"keda-metrics-apiserver-77857b7b98-crzsn\" (UID: \"424a4a19-21fe-47d0-bfd7-45122285a714\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-crzsn"
Sep 17 16:47:27.563716 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.563676    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/424a4a19-21fe-47d0-bfd7-45122285a714-certificates\") pod \"keda-metrics-apiserver-77857b7b98-crzsn\" (UID: \"424a4a19-21fe-47d0-bfd7-45122285a714\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-crzsn"
Sep 17 16:47:27.563921 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.563751    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svrq8\" (UniqueName: \"kubernetes.io/projected/424a4a19-21fe-47d0-bfd7-45122285a714-kube-api-access-svrq8\") pod \"keda-metrics-apiserver-77857b7b98-crzsn\" (UID: \"424a4a19-21fe-47d0-bfd7-45122285a714\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-crzsn"
Sep 17 16:47:27.563921 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.563790    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/424a4a19-21fe-47d0-bfd7-45122285a714-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-crzsn\" (UID: \"424a4a19-21fe-47d0-bfd7-45122285a714\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-crzsn"
Sep 17 16:47:27.563921 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:47:27.563831    2586 secret.go:281] references non-existent secret key: tls.crt
Sep 17 16:47:27.563921 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:47:27.563849    2586 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Sep 17 16:47:27.563921 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:47:27.563899    2586 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-77857b7b98-crzsn: references non-existent secret key: tls.crt
Sep 17 16:47:27.564121 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:47:27.563962    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/424a4a19-21fe-47d0-bfd7-45122285a714-certificates podName:424a4a19-21fe-47d0-bfd7-45122285a714 nodeName:}" failed. No retries permitted until 2026-09-17 16:47:28.063945499 +0000 UTC m=+318.765024154 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/424a4a19-21fe-47d0-bfd7-45122285a714-certificates") pod "keda-metrics-apiserver-77857b7b98-crzsn" (UID: "424a4a19-21fe-47d0-bfd7-45122285a714") : references non-existent secret key: tls.crt
Sep 17 16:47:27.564224 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.564194    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/424a4a19-21fe-47d0-bfd7-45122285a714-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-crzsn\" (UID: \"424a4a19-21fe-47d0-bfd7-45122285a714\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-crzsn"
Sep 17 16:47:27.573219 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.573191    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svrq8\" (UniqueName: \"kubernetes.io/projected/424a4a19-21fe-47d0-bfd7-45122285a714-kube-api-access-svrq8\") pod \"keda-metrics-apiserver-77857b7b98-crzsn\" (UID: \"424a4a19-21fe-47d0-bfd7-45122285a714\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-crzsn"
Sep 17 16:47:27.741051 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.741013    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-6knr2"]
Sep 17 16:47:27.744252 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.744234    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-6c4d864b86-6knr2"
Sep 17 16:47:27.746808 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.746786    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Sep 17 16:47:27.753111 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.753086    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-6knr2"]
Sep 17 16:47:27.765522 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.765485    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/1421cd1e-dde5-4eb7-bbbf-4a89f59cb475-certificates\") pod \"keda-operator-579c8958f7-hxhdw\" (UID: \"1421cd1e-dde5-4eb7-bbbf-4a89f59cb475\") " pod="openshift-keda/keda-operator-579c8958f7-hxhdw"
Sep 17 16:47:27.768206 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.768187    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/1421cd1e-dde5-4eb7-bbbf-4a89f59cb475-certificates\") pod \"keda-operator-579c8958f7-hxhdw\" (UID: \"1421cd1e-dde5-4eb7-bbbf-4a89f59cb475\") " pod="openshift-keda/keda-operator-579c8958f7-hxhdw"
Sep 17 16:47:27.867072 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.866971    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/53cc6507-46c8-4012-bb1f-eaef1b111037-certificates\") pod \"keda-admission-6c4d864b86-6knr2\" (UID: \"53cc6507-46c8-4012-bb1f-eaef1b111037\") " pod="openshift-keda/keda-admission-6c4d864b86-6knr2"
Sep 17 16:47:27.867072 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.867023    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r74kk\" (UniqueName: \"kubernetes.io/projected/53cc6507-46c8-4012-bb1f-eaef1b111037-kube-api-access-r74kk\") pod \"keda-admission-6c4d864b86-6knr2\" (UID: \"53cc6507-46c8-4012-bb1f-eaef1b111037\") " pod="openshift-keda/keda-admission-6c4d864b86-6knr2"
Sep 17 16:47:27.888004 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.887955    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-579c8958f7-hxhdw"
Sep 17 16:47:27.968663 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.968626    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/53cc6507-46c8-4012-bb1f-eaef1b111037-certificates\") pod \"keda-admission-6c4d864b86-6knr2\" (UID: \"53cc6507-46c8-4012-bb1f-eaef1b111037\") " pod="openshift-keda/keda-admission-6c4d864b86-6knr2"
Sep 17 16:47:27.968827 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.968677    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r74kk\" (UniqueName: \"kubernetes.io/projected/53cc6507-46c8-4012-bb1f-eaef1b111037-kube-api-access-r74kk\") pod \"keda-admission-6c4d864b86-6knr2\" (UID: \"53cc6507-46c8-4012-bb1f-eaef1b111037\") " pod="openshift-keda/keda-admission-6c4d864b86-6knr2"
Sep 17 16:47:27.969154 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:47:27.968997    2586 projected.go:266] Couldn't get secret openshift-keda/keda-admission-webhooks-certs: secret "keda-admission-webhooks-certs" not found
Sep 17 16:47:27.969154 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:47:27.969025    2586 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-admission-6c4d864b86-6knr2: secret "keda-admission-webhooks-certs" not found
Sep 17 16:47:27.969154 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:47:27.969088    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/53cc6507-46c8-4012-bb1f-eaef1b111037-certificates podName:53cc6507-46c8-4012-bb1f-eaef1b111037 nodeName:}" failed. No retries permitted until 2026-09-17 16:47:28.46907268 +0000 UTC m=+319.170151321 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/53cc6507-46c8-4012-bb1f-eaef1b111037-certificates") pod "keda-admission-6c4d864b86-6knr2" (UID: "53cc6507-46c8-4012-bb1f-eaef1b111037") : secret "keda-admission-webhooks-certs" not found
Sep 17 16:47:27.978821 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:27.978759    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r74kk\" (UniqueName: \"kubernetes.io/projected/53cc6507-46c8-4012-bb1f-eaef1b111037-kube-api-access-r74kk\") pod \"keda-admission-6c4d864b86-6knr2\" (UID: \"53cc6507-46c8-4012-bb1f-eaef1b111037\") " pod="openshift-keda/keda-admission-6c4d864b86-6knr2"
Sep 17 16:47:28.069680 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:28.069155    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/424a4a19-21fe-47d0-bfd7-45122285a714-certificates\") pod \"keda-metrics-apiserver-77857b7b98-crzsn\" (UID: \"424a4a19-21fe-47d0-bfd7-45122285a714\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-crzsn"
Sep 17 16:47:28.072389 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:28.072360    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/424a4a19-21fe-47d0-bfd7-45122285a714-certificates\") pod \"keda-metrics-apiserver-77857b7b98-crzsn\" (UID: \"424a4a19-21fe-47d0-bfd7-45122285a714\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-crzsn"
Sep 17 16:47:28.169728 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:28.168523    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-579c8958f7-hxhdw"]
Sep 17 16:47:28.171250 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:47:28.171218    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1421cd1e_dde5_4eb7_bbbf_4a89f59cb475.slice/crio-bc63bb2a44a36208013ac807303916553d6fe2c5300147e84e6978814a02d0c3 WatchSource:0}: Error finding container bc63bb2a44a36208013ac807303916553d6fe2c5300147e84e6978814a02d0c3: Status 404 returned error can't find the container with id bc63bb2a44a36208013ac807303916553d6fe2c5300147e84e6978814a02d0c3
Sep 17 16:47:28.211741 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:28.211707    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-crzsn"
Sep 17 16:47:28.473235 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:28.473152    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/53cc6507-46c8-4012-bb1f-eaef1b111037-certificates\") pod \"keda-admission-6c4d864b86-6knr2\" (UID: \"53cc6507-46c8-4012-bb1f-eaef1b111037\") " pod="openshift-keda/keda-admission-6c4d864b86-6knr2"
Sep 17 16:47:28.475961 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:28.475937    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/53cc6507-46c8-4012-bb1f-eaef1b111037-certificates\") pod \"keda-admission-6c4d864b86-6knr2\" (UID: \"53cc6507-46c8-4012-bb1f-eaef1b111037\") " pod="openshift-keda/keda-admission-6c4d864b86-6knr2"
Sep 17 16:47:28.501369 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:28.501338    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-crzsn"]
Sep 17 16:47:28.506679 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:47:28.506642    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod424a4a19_21fe_47d0_bfd7_45122285a714.slice/crio-9b41a102d4921c003b3d4f1c5f1fadb98ec7656c24c8528da93c67ee3a7b0888 WatchSource:0}: Error finding container 9b41a102d4921c003b3d4f1c5f1fadb98ec7656c24c8528da93c67ee3a7b0888: Status 404 returned error can't find the container with id 9b41a102d4921c003b3d4f1c5f1fadb98ec7656c24c8528da93c67ee3a7b0888
Sep 17 16:47:28.655730 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:28.655693    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-6c4d864b86-6knr2"
Sep 17 16:47:28.663030 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:28.662997    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-crzsn" event={"ID":"424a4a19-21fe-47d0-bfd7-45122285a714","Type":"ContainerStarted","Data":"9b41a102d4921c003b3d4f1c5f1fadb98ec7656c24c8528da93c67ee3a7b0888"}
Sep 17 16:47:28.664299 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:28.664273    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-579c8958f7-hxhdw" event={"ID":"1421cd1e-dde5-4eb7-bbbf-4a89f59cb475","Type":"ContainerStarted","Data":"bc63bb2a44a36208013ac807303916553d6fe2c5300147e84e6978814a02d0c3"}
Sep 17 16:47:28.939993 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:28.939955    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-6knr2"]
Sep 17 16:47:28.944497 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:47:28.944464    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53cc6507_46c8_4012_bb1f_eaef1b111037.slice/crio-1f4c292bef299702c9b589b1433e8dcd6e35c2d1420a946df378d26baa0e0853 WatchSource:0}: Error finding container 1f4c292bef299702c9b589b1433e8dcd6e35c2d1420a946df378d26baa0e0853: Status 404 returned error can't find the container with id 1f4c292bef299702c9b589b1433e8dcd6e35c2d1420a946df378d26baa0e0853
Sep 17 16:47:29.670250 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:29.670209    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-6c4d864b86-6knr2" event={"ID":"53cc6507-46c8-4012-bb1f-eaef1b111037","Type":"ContainerStarted","Data":"1f4c292bef299702c9b589b1433e8dcd6e35c2d1420a946df378d26baa0e0853"}
Sep 17 16:47:32.687262 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:32.687153    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-crzsn" event={"ID":"424a4a19-21fe-47d0-bfd7-45122285a714","Type":"ContainerStarted","Data":"b8609ff793e4079e317157ff07ff48cf43e7a5c840b840a023b12091344b4556"}
Sep 17 16:47:32.687699 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:32.687298    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-crzsn"
Sep 17 16:47:32.688908 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:32.688865    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-579c8958f7-hxhdw" event={"ID":"1421cd1e-dde5-4eb7-bbbf-4a89f59cb475","Type":"ContainerStarted","Data":"dda3fa6b6df2ff79b54c286483dfa559f5d42a2bfac5879bdc851548064aa9aa"}
Sep 17 16:47:32.689010 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:32.688950    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-579c8958f7-hxhdw"
Sep 17 16:47:32.690502 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:32.690472    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-6c4d864b86-6knr2" event={"ID":"53cc6507-46c8-4012-bb1f-eaef1b111037","Type":"ContainerStarted","Data":"8d0c93cf65191aa69f3d04baa133dcf29c6eb5d14990c59b329ac473f52deb2f"}
Sep 17 16:47:32.690632 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:32.690617    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-6c4d864b86-6knr2"
Sep 17 16:47:32.704980 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:32.704927    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-crzsn" podStartSLOduration=1.7670928799999999 podStartE2EDuration="5.704910744s" totalImagesPullingTime="3.937817864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:47:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:47:32.703895857 +0000 UTC m=+323.404974511" watchObservedRunningTime="2026-09-17 16:47:32.704910744 +0000 UTC m=+323.405989407"
Sep 17 16:47:32.721154 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:32.721091    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-579c8958f7-hxhdw" podStartSLOduration=2.453635185 podStartE2EDuration="6.72106646s" totalImagesPullingTime="4.267431275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:47:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:47:32.720075948 +0000 UTC m=+323.421154611" watchObservedRunningTime="2026-09-17 16:47:32.72106646 +0000 UTC m=+323.422145123"
Sep 17 16:47:32.737221 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:32.737169    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-6c4d864b86-6knr2" podStartSLOduration=2.240227758 podStartE2EDuration="5.737152171s" totalImagesPullingTime="3.496924413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:47:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:47:32.735797795 +0000 UTC m=+323.436876481" watchObservedRunningTime="2026-09-17 16:47:32.737152171 +0000 UTC m=+323.438230832"
Sep 17 16:47:43.699438 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:43.699409    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-crzsn"
Sep 17 16:47:47.659644 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:47.659614    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-lzk5t"
Sep 17 16:47:53.697611 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:53.697530    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-579c8958f7-hxhdw"
Sep 17 16:47:53.698174 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:47:53.697612    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-6c4d864b86-6knr2"
Sep 17 16:48:26.085362 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:26.085310    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-79cb5fcc48-96qkf"]
Sep 17 16:48:26.088538 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:26.088521    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79cb5fcc48-96qkf"
Sep 17 16:48:26.103217 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:26.103188    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79cb5fcc48-96qkf"]
Sep 17 16:48:26.168339 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:26.168299    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25e1699a-db36-41fa-8ab9-03b02e86d1cb-service-ca\") pod \"console-79cb5fcc48-96qkf\" (UID: \"25e1699a-db36-41fa-8ab9-03b02e86d1cb\") " pod="openshift-console/console-79cb5fcc48-96qkf"
Sep 17 16:48:26.168339 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:26.168342    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25e1699a-db36-41fa-8ab9-03b02e86d1cb-console-oauth-config\") pod \"console-79cb5fcc48-96qkf\" (UID: \"25e1699a-db36-41fa-8ab9-03b02e86d1cb\") " pod="openshift-console/console-79cb5fcc48-96qkf"
Sep 17 16:48:26.168591 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:26.168385    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25e1699a-db36-41fa-8ab9-03b02e86d1cb-oauth-serving-cert\") pod \"console-79cb5fcc48-96qkf\" (UID: \"25e1699a-db36-41fa-8ab9-03b02e86d1cb\") " pod="openshift-console/console-79cb5fcc48-96qkf"
Sep 17 16:48:26.168591 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:26.168407    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cwh6\" (UniqueName: \"kubernetes.io/projected/25e1699a-db36-41fa-8ab9-03b02e86d1cb-kube-api-access-2cwh6\") pod \"console-79cb5fcc48-96qkf\" (UID: \"25e1699a-db36-41fa-8ab9-03b02e86d1cb\") " pod="openshift-console/console-79cb5fcc48-96qkf"
Sep 17 16:48:26.168591 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:26.168468    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25e1699a-db36-41fa-8ab9-03b02e86d1cb-console-config\") pod \"console-79cb5fcc48-96qkf\" (UID: \"25e1699a-db36-41fa-8ab9-03b02e86d1cb\") " pod="openshift-console/console-79cb5fcc48-96qkf"
Sep 17 16:48:26.168591 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:26.168493    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25e1699a-db36-41fa-8ab9-03b02e86d1cb-console-serving-cert\") pod \"console-79cb5fcc48-96qkf\" (UID: \"25e1699a-db36-41fa-8ab9-03b02e86d1cb\") " pod="openshift-console/console-79cb5fcc48-96qkf"
Sep 17 16:48:26.168591 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:26.168516    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25e1699a-db36-41fa-8ab9-03b02e86d1cb-trusted-ca-bundle\") pod \"console-79cb5fcc48-96qkf\" (UID: \"25e1699a-db36-41fa-8ab9-03b02e86d1cb\") " pod="openshift-console/console-79cb5fcc48-96qkf"
Sep 17 16:48:26.269157 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:26.269115    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25e1699a-db36-41fa-8ab9-03b02e86d1cb-console-config\") pod \"console-79cb5fcc48-96qkf\" (UID: \"25e1699a-db36-41fa-8ab9-03b02e86d1cb\") " pod="openshift-console/console-79cb5fcc48-96qkf"
Sep 17 16:48:26.269157 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:26.269160    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25e1699a-db36-41fa-8ab9-03b02e86d1cb-console-serving-cert\") pod \"console-79cb5fcc48-96qkf\" (UID: \"25e1699a-db36-41fa-8ab9-03b02e86d1cb\") " pod="openshift-console/console-79cb5fcc48-96qkf"
Sep 17 16:48:26.269408 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:26.269183    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25e1699a-db36-41fa-8ab9-03b02e86d1cb-trusted-ca-bundle\") pod \"console-79cb5fcc48-96qkf\" (UID: \"25e1699a-db36-41fa-8ab9-03b02e86d1cb\") " pod="openshift-console/console-79cb5fcc48-96qkf"
Sep 17 16:48:26.269408 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:26.269231    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25e1699a-db36-41fa-8ab9-03b02e86d1cb-service-ca\") pod \"console-79cb5fcc48-96qkf\" (UID: \"25e1699a-db36-41fa-8ab9-03b02e86d1cb\") " pod="openshift-console/console-79cb5fcc48-96qkf"
Sep 17 16:48:26.269408 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:26.269255    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25e1699a-db36-41fa-8ab9-03b02e86d1cb-console-oauth-config\") pod \"console-79cb5fcc48-96qkf\" (UID: \"25e1699a-db36-41fa-8ab9-03b02e86d1cb\") " pod="openshift-console/console-79cb5fcc48-96qkf"
Sep 17 16:48:26.269408 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:26.269291    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25e1699a-db36-41fa-8ab9-03b02e86d1cb-oauth-serving-cert\") pod \"console-79cb5fcc48-96qkf\" (UID: \"25e1699a-db36-41fa-8ab9-03b02e86d1cb\") " pod="openshift-console/console-79cb5fcc48-96qkf"
Sep 17 16:48:26.269408 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:26.269313    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cwh6\" (UniqueName: \"kubernetes.io/projected/25e1699a-db36-41fa-8ab9-03b02e86d1cb-kube-api-access-2cwh6\") pod \"console-79cb5fcc48-96qkf\" (UID: \"25e1699a-db36-41fa-8ab9-03b02e86d1cb\") " pod="openshift-console/console-79cb5fcc48-96qkf"
Sep 17 16:48:26.269970 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:26.269942    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25e1699a-db36-41fa-8ab9-03b02e86d1cb-console-config\") pod \"console-79cb5fcc48-96qkf\" (UID: \"25e1699a-db36-41fa-8ab9-03b02e86d1cb\") " pod="openshift-console/console-79cb5fcc48-96qkf"
Sep 17 16:48:26.270094 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:26.270028    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25e1699a-db36-41fa-8ab9-03b02e86d1cb-service-ca\") pod \"console-79cb5fcc48-96qkf\" (UID: \"25e1699a-db36-41fa-8ab9-03b02e86d1cb\") " pod="openshift-console/console-79cb5fcc48-96qkf"
Sep 17 16:48:26.270094 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:26.270049    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25e1699a-db36-41fa-8ab9-03b02e86d1cb-oauth-serving-cert\") pod \"console-79cb5fcc48-96qkf\" (UID: \"25e1699a-db36-41fa-8ab9-03b02e86d1cb\") " pod="openshift-console/console-79cb5fcc48-96qkf"
Sep 17 16:48:26.270181 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:26.270155    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25e1699a-db36-41fa-8ab9-03b02e86d1cb-trusted-ca-bundle\") pod \"console-79cb5fcc48-96qkf\" (UID: \"25e1699a-db36-41fa-8ab9-03b02e86d1cb\") " pod="openshift-console/console-79cb5fcc48-96qkf"
Sep 17 16:48:26.272442 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:26.272416    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25e1699a-db36-41fa-8ab9-03b02e86d1cb-console-serving-cert\") pod \"console-79cb5fcc48-96qkf\" (UID: \"25e1699a-db36-41fa-8ab9-03b02e86d1cb\") " pod="openshift-console/console-79cb5fcc48-96qkf"
Sep 17 16:48:26.272533 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:26.272423    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25e1699a-db36-41fa-8ab9-03b02e86d1cb-console-oauth-config\") pod \"console-79cb5fcc48-96qkf\" (UID: \"25e1699a-db36-41fa-8ab9-03b02e86d1cb\") " pod="openshift-console/console-79cb5fcc48-96qkf"
Sep 17 16:48:26.283803 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:26.283765    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cwh6\" (UniqueName: \"kubernetes.io/projected/25e1699a-db36-41fa-8ab9-03b02e86d1cb-kube-api-access-2cwh6\") pod \"console-79cb5fcc48-96qkf\" (UID: \"25e1699a-db36-41fa-8ab9-03b02e86d1cb\") " pod="openshift-console/console-79cb5fcc48-96qkf"
Sep 17 16:48:26.397817 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:26.397726    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79cb5fcc48-96qkf"
Sep 17 16:48:26.675403 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:26.675372    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79cb5fcc48-96qkf"]
Sep 17 16:48:26.677853 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:48:26.677828    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25e1699a_db36_41fa_8ab9_03b02e86d1cb.slice/crio-553d0add2b2d07b03d3c22811c7c2ed24e4322c4c602e679118abfe25653270a WatchSource:0}: Error finding container 553d0add2b2d07b03d3c22811c7c2ed24e4322c4c602e679118abfe25653270a: Status 404 returned error can't find the container with id 553d0add2b2d07b03d3c22811c7c2ed24e4322c4c602e679118abfe25653270a
Sep 17 16:48:26.917207 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:26.917163    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79cb5fcc48-96qkf" event={"ID":"25e1699a-db36-41fa-8ab9-03b02e86d1cb","Type":"ContainerStarted","Data":"d9ad180ce1f62eee8781a056b6441191347792948ff2cd2ffe89ecdb0deaef47"}
Sep 17 16:48:26.917207 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:26.917215    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79cb5fcc48-96qkf" event={"ID":"25e1699a-db36-41fa-8ab9-03b02e86d1cb","Type":"ContainerStarted","Data":"553d0add2b2d07b03d3c22811c7c2ed24e4322c4c602e679118abfe25653270a"}
Sep 17 16:48:26.939121 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:26.939019    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-79cb5fcc48-96qkf" podStartSLOduration=0.939000128 podStartE2EDuration="939.000128ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:48:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:48:26.938421156 +0000 UTC m=+377.639499823" watchObservedRunningTime="2026-09-17 16:48:26.939000128 +0000 UTC m=+377.640078822"
Sep 17 16:48:36.397972 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:36.397941    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-79cb5fcc48-96qkf"
Sep 17 16:48:36.398354 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:36.397997    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-79cb5fcc48-96qkf"
Sep 17 16:48:36.403138 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:36.403114    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-79cb5fcc48-96qkf"
Sep 17 16:48:36.962614 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:36.962586    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-79cb5fcc48-96qkf"
Sep 17 16:48:37.011006 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:37.010973    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cc6456699-sfwst"]
Sep 17 16:48:47.457299 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:47.457267    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-846cf5d849-hdzwj"]
Sep 17 16:48:47.460907 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:47.460863    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-846cf5d849-hdzwj"
Sep 17 16:48:47.463750 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:47.463730    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Sep 17 16:48:47.464820 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:47.464802    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-gfx9n\""
Sep 17 16:48:47.465163 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:47.465143    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Sep 17 16:48:47.465269 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:47.465162    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Sep 17 16:48:47.467544 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:47.467522    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-645f9c8fd6-p5dqp"]
Sep 17 16:48:47.470600 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:47.470583    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-645f9c8fd6-p5dqp"
Sep 17 16:48:47.473173 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:47.473148    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-qkn2k\""
Sep 17 16:48:47.474029 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:47.474009    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Sep 17 16:48:47.475112 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:47.475084    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-846cf5d849-hdzwj"]
Sep 17 16:48:47.483579 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:47.483549    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-645f9c8fd6-p5dqp"]
Sep 17 16:48:47.554205 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:47.554163    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b198ab34-0f7a-4cfe-8461-6dfc2b682aa3-cert\") pod \"kserve-controller-manager-846cf5d849-hdzwj\" (UID: \"b198ab34-0f7a-4cfe-8461-6dfc2b682aa3\") " pod="kserve/kserve-controller-manager-846cf5d849-hdzwj"
Sep 17 16:48:47.554205 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:47.554211    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpnpq\" (UniqueName: \"kubernetes.io/projected/b198ab34-0f7a-4cfe-8461-6dfc2b682aa3-kube-api-access-qpnpq\") pod \"kserve-controller-manager-846cf5d849-hdzwj\" (UID: \"b198ab34-0f7a-4cfe-8461-6dfc2b682aa3\") " pod="kserve/kserve-controller-manager-846cf5d849-hdzwj"
Sep 17 16:48:47.554427 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:47.554230    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ddcd98b1-18b4-42eb-b229-1423d2740ecd-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-p5dqp\" (UID: \"ddcd98b1-18b4-42eb-b229-1423d2740ecd\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-p5dqp"
Sep 17 16:48:47.554427 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:47.554257    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8g47\" (UniqueName: \"kubernetes.io/projected/ddcd98b1-18b4-42eb-b229-1423d2740ecd-kube-api-access-v8g47\") pod \"llmisvc-controller-manager-645f9c8fd6-p5dqp\" (UID: \"ddcd98b1-18b4-42eb-b229-1423d2740ecd\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-p5dqp"
Sep 17 16:48:47.654842 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:47.654795    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8g47\" (UniqueName: \"kubernetes.io/projected/ddcd98b1-18b4-42eb-b229-1423d2740ecd-kube-api-access-v8g47\") pod \"llmisvc-controller-manager-645f9c8fd6-p5dqp\" (UID: \"ddcd98b1-18b4-42eb-b229-1423d2740ecd\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-p5dqp"
Sep 17 16:48:47.655064 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:47.654933    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b198ab34-0f7a-4cfe-8461-6dfc2b682aa3-cert\") pod \"kserve-controller-manager-846cf5d849-hdzwj\" (UID: \"b198ab34-0f7a-4cfe-8461-6dfc2b682aa3\") " pod="kserve/kserve-controller-manager-846cf5d849-hdzwj"
Sep 17 16:48:47.655064 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:47.654970    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpnpq\" (UniqueName: \"kubernetes.io/projected/b198ab34-0f7a-4cfe-8461-6dfc2b682aa3-kube-api-access-qpnpq\") pod \"kserve-controller-manager-846cf5d849-hdzwj\" (UID: \"b198ab34-0f7a-4cfe-8461-6dfc2b682aa3\") " pod="kserve/kserve-controller-manager-846cf5d849-hdzwj"
Sep 17 16:48:47.655064 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:47.654992    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ddcd98b1-18b4-42eb-b229-1423d2740ecd-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-p5dqp\" (UID: \"ddcd98b1-18b4-42eb-b229-1423d2740ecd\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-p5dqp"
Sep 17 16:48:47.655253 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:48:47.655086    2586 secret.go:189] Couldn't get secret kserve/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 17 16:48:47.655253 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:48:47.655172    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b198ab34-0f7a-4cfe-8461-6dfc2b682aa3-cert podName:b198ab34-0f7a-4cfe-8461-6dfc2b682aa3 nodeName:}" failed. No retries permitted until 2026-09-17 16:48:48.155153755 +0000 UTC m=+398.856232401 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b198ab34-0f7a-4cfe-8461-6dfc2b682aa3-cert") pod "kserve-controller-manager-846cf5d849-hdzwj" (UID: "b198ab34-0f7a-4cfe-8461-6dfc2b682aa3") : secret "kserve-webhook-server-cert" not found
Sep 17 16:48:47.657841 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:47.657816    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ddcd98b1-18b4-42eb-b229-1423d2740ecd-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-p5dqp\" (UID: \"ddcd98b1-18b4-42eb-b229-1423d2740ecd\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-p5dqp"
Sep 17 16:48:47.664465 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:47.664441    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpnpq\" (UniqueName: \"kubernetes.io/projected/b198ab34-0f7a-4cfe-8461-6dfc2b682aa3-kube-api-access-qpnpq\") pod \"kserve-controller-manager-846cf5d849-hdzwj\" (UID: \"b198ab34-0f7a-4cfe-8461-6dfc2b682aa3\") " pod="kserve/kserve-controller-manager-846cf5d849-hdzwj"
Sep 17 16:48:47.665043 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:47.665024    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8g47\" (UniqueName: \"kubernetes.io/projected/ddcd98b1-18b4-42eb-b229-1423d2740ecd-kube-api-access-v8g47\") pod \"llmisvc-controller-manager-645f9c8fd6-p5dqp\" (UID: \"ddcd98b1-18b4-42eb-b229-1423d2740ecd\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-p5dqp"
Sep 17 16:48:47.789508 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:47.789476    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-645f9c8fd6-p5dqp"
Sep 17 16:48:48.059833 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:48.059795    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-645f9c8fd6-p5dqp"]
Sep 17 16:48:48.063331 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:48:48.063304    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podddcd98b1_18b4_42eb_b229_1423d2740ecd.slice/crio-e1c61a12145710fb8a3f1433fd1fa6cfe029d31214669e9100344662f90151e7 WatchSource:0}: Error finding container e1c61a12145710fb8a3f1433fd1fa6cfe029d31214669e9100344662f90151e7: Status 404 returned error can't find the container with id e1c61a12145710fb8a3f1433fd1fa6cfe029d31214669e9100344662f90151e7
Sep 17 16:48:48.159735 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:48.159695    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b198ab34-0f7a-4cfe-8461-6dfc2b682aa3-cert\") pod \"kserve-controller-manager-846cf5d849-hdzwj\" (UID: \"b198ab34-0f7a-4cfe-8461-6dfc2b682aa3\") " pod="kserve/kserve-controller-manager-846cf5d849-hdzwj"
Sep 17 16:48:48.162322 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:48.162301    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b198ab34-0f7a-4cfe-8461-6dfc2b682aa3-cert\") pod \"kserve-controller-manager-846cf5d849-hdzwj\" (UID: \"b198ab34-0f7a-4cfe-8461-6dfc2b682aa3\") " pod="kserve/kserve-controller-manager-846cf5d849-hdzwj"
Sep 17 16:48:48.372322 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:48.372220    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-846cf5d849-hdzwj"
Sep 17 16:48:48.643801 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:48.643774    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-846cf5d849-hdzwj"]
Sep 17 16:48:48.646248 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:48:48.646215    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb198ab34_0f7a_4cfe_8461_6dfc2b682aa3.slice/crio-7c8b1b8fd4d118240cf9684c76789762c54d97c2533e36f23124b03ebdd44ad7 WatchSource:0}: Error finding container 7c8b1b8fd4d118240cf9684c76789762c54d97c2533e36f23124b03ebdd44ad7: Status 404 returned error can't find the container with id 7c8b1b8fd4d118240cf9684c76789762c54d97c2533e36f23124b03ebdd44ad7
Sep 17 16:48:49.009619 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:49.009526    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-846cf5d849-hdzwj" event={"ID":"b198ab34-0f7a-4cfe-8461-6dfc2b682aa3","Type":"ContainerStarted","Data":"7c8b1b8fd4d118240cf9684c76789762c54d97c2533e36f23124b03ebdd44ad7"}
Sep 17 16:48:49.011904 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:49.011855    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-645f9c8fd6-p5dqp" event={"ID":"ddcd98b1-18b4-42eb-b229-1423d2740ecd","Type":"ContainerStarted","Data":"e1c61a12145710fb8a3f1433fd1fa6cfe029d31214669e9100344662f90151e7"}
Sep 17 16:48:52.030495 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:52.030459    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-846cf5d849-hdzwj" event={"ID":"b198ab34-0f7a-4cfe-8461-6dfc2b682aa3","Type":"ContainerStarted","Data":"1aa380ec96f6455c09be528ca93f45512763accca33a061632979edbe5541ee7"}
Sep 17 16:48:52.030957 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:52.030584    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-846cf5d849-hdzwj"
Sep 17 16:48:52.031980 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:52.031958    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-645f9c8fd6-p5dqp" event={"ID":"ddcd98b1-18b4-42eb-b229-1423d2740ecd","Type":"ContainerStarted","Data":"a50085e1316e3ef68dccf4698bf0249e8b78356ed2cc4383481c278116d355e5"}
Sep 17 16:48:52.032079 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:52.032005    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-645f9c8fd6-p5dqp"
Sep 17 16:48:52.051118 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:52.050991    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-846cf5d849-hdzwj" podStartSLOduration=2.189377682 podStartE2EDuration="5.050968399s" totalImagesPullingTime="2.861590717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:48:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:48:52.050789716 +0000 UTC m=+402.751868379" watchObservedRunningTime="2026-09-17 16:48:52.050968399 +0000 UTC m=+402.752047063"
Sep 17 16:48:52.068192 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:48:52.068136    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-645f9c8fd6-p5dqp" podStartSLOduration=1.6265277299999998 podStartE2EDuration="5.068116227s" totalImagesPullingTime="3.441588497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:48:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:48:52.067494601 +0000 UTC m=+402.768573263" watchObservedRunningTime="2026-09-17 16:48:52.068116227 +0000 UTC m=+402.769194888"
Sep 17 16:49:02.031331 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:02.031284    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6cc6456699-sfwst" podUID="fc010cbe-909b-4b0a-b293-92627cb28412" containerName="console" containerID="cri-o://ae4eb31bd436d24b8bdd9f7f21d34e11ddfdc5e184d74180902879df0e8a4bc4" gracePeriod=15
Sep 17 16:49:02.271237 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:02.271209    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cc6456699-sfwst"
Sep 17 16:49:02.383805 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:02.383714    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc010cbe-909b-4b0a-b293-92627cb28412-service-ca\") pod \"fc010cbe-909b-4b0a-b293-92627cb28412\" (UID: \"fc010cbe-909b-4b0a-b293-92627cb28412\") "
Sep 17 16:49:02.383805 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:02.383756    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h682t\" (UniqueName: \"kubernetes.io/projected/fc010cbe-909b-4b0a-b293-92627cb28412-kube-api-access-h682t\") pod \"fc010cbe-909b-4b0a-b293-92627cb28412\" (UID: \"fc010cbe-909b-4b0a-b293-92627cb28412\") "
Sep 17 16:49:02.383805 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:02.383776    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc010cbe-909b-4b0a-b293-92627cb28412-oauth-serving-cert\") pod \"fc010cbe-909b-4b0a-b293-92627cb28412\" (UID: \"fc010cbe-909b-4b0a-b293-92627cb28412\") "
Sep 17 16:49:02.384117 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:02.383821    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc010cbe-909b-4b0a-b293-92627cb28412-console-config\") pod \"fc010cbe-909b-4b0a-b293-92627cb28412\" (UID: \"fc010cbe-909b-4b0a-b293-92627cb28412\") "
Sep 17 16:49:02.384117 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:02.383944    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc010cbe-909b-4b0a-b293-92627cb28412-console-serving-cert\") pod \"fc010cbe-909b-4b0a-b293-92627cb28412\" (UID: \"fc010cbe-909b-4b0a-b293-92627cb28412\") "
Sep 17 16:49:02.384117 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:02.384016    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc010cbe-909b-4b0a-b293-92627cb28412-trusted-ca-bundle\") pod \"fc010cbe-909b-4b0a-b293-92627cb28412\" (UID: \"fc010cbe-909b-4b0a-b293-92627cb28412\") "
Sep 17 16:49:02.384117 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:02.384062    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc010cbe-909b-4b0a-b293-92627cb28412-console-oauth-config\") pod \"fc010cbe-909b-4b0a-b293-92627cb28412\" (UID: \"fc010cbe-909b-4b0a-b293-92627cb28412\") "
Sep 17 16:49:02.384323 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:02.384186    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc010cbe-909b-4b0a-b293-92627cb28412-service-ca" (OuterVolumeSpecName: "service-ca") pod "fc010cbe-909b-4b0a-b293-92627cb28412" (UID: "fc010cbe-909b-4b0a-b293-92627cb28412"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:49:02.384323 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:02.384196    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc010cbe-909b-4b0a-b293-92627cb28412-console-config" (OuterVolumeSpecName: "console-config") pod "fc010cbe-909b-4b0a-b293-92627cb28412" (UID: "fc010cbe-909b-4b0a-b293-92627cb28412"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:49:02.384323 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:02.384301    2586 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc010cbe-909b-4b0a-b293-92627cb28412-service-ca\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:49:02.384323 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:02.384319    2586 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc010cbe-909b-4b0a-b293-92627cb28412-console-config\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:49:02.384480 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:02.384377    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc010cbe-909b-4b0a-b293-92627cb28412-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fc010cbe-909b-4b0a-b293-92627cb28412" (UID: "fc010cbe-909b-4b0a-b293-92627cb28412"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:49:02.384514 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:02.384474    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc010cbe-909b-4b0a-b293-92627cb28412-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fc010cbe-909b-4b0a-b293-92627cb28412" (UID: "fc010cbe-909b-4b0a-b293-92627cb28412"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:49:02.386137 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:02.386109    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc010cbe-909b-4b0a-b293-92627cb28412-kube-api-access-h682t" (OuterVolumeSpecName: "kube-api-access-h682t") pod "fc010cbe-909b-4b0a-b293-92627cb28412" (UID: "fc010cbe-909b-4b0a-b293-92627cb28412"). InnerVolumeSpecName "kube-api-access-h682t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:49:02.386259 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:02.386172    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc010cbe-909b-4b0a-b293-92627cb28412-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fc010cbe-909b-4b0a-b293-92627cb28412" (UID: "fc010cbe-909b-4b0a-b293-92627cb28412"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:49:02.386302 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:02.386275    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc010cbe-909b-4b0a-b293-92627cb28412-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fc010cbe-909b-4b0a-b293-92627cb28412" (UID: "fc010cbe-909b-4b0a-b293-92627cb28412"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:49:02.485820 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:02.485779    2586 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc010cbe-909b-4b0a-b293-92627cb28412-console-serving-cert\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:49:02.485820 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:02.485817    2586 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc010cbe-909b-4b0a-b293-92627cb28412-trusted-ca-bundle\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:49:02.486100 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:02.485833    2586 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc010cbe-909b-4b0a-b293-92627cb28412-console-oauth-config\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:49:02.486100 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:02.485847    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h682t\" (UniqueName: \"kubernetes.io/projected/fc010cbe-909b-4b0a-b293-92627cb28412-kube-api-access-h682t\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:49:02.486100 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:02.485860    2586 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc010cbe-909b-4b0a-b293-92627cb28412-oauth-serving-cert\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:49:03.079692 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:03.079662    2586 generic.go:356] "Generic (PLEG): container finished" podID="fc010cbe-909b-4b0a-b293-92627cb28412" containerID="ae4eb31bd436d24b8bdd9f7f21d34e11ddfdc5e184d74180902879df0e8a4bc4" exitCode=0
Sep 17 16:49:03.080129 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:03.079710    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cc6456699-sfwst"
Sep 17 16:49:03.080129 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:03.079758    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cc6456699-sfwst" event={"ID":"fc010cbe-909b-4b0a-b293-92627cb28412","Type":"ContainerDied","Data":"ae4eb31bd436d24b8bdd9f7f21d34e11ddfdc5e184d74180902879df0e8a4bc4"}
Sep 17 16:49:03.080129 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:03.079808    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cc6456699-sfwst" event={"ID":"fc010cbe-909b-4b0a-b293-92627cb28412","Type":"ContainerDied","Data":"91eb9d61dc734adcc57086da5958d072c2a859d963c495550e7b7c27908af229"}
Sep 17 16:49:03.080129 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:03.079829    2586 scope.go:117] "RemoveContainer" containerID="ae4eb31bd436d24b8bdd9f7f21d34e11ddfdc5e184d74180902879df0e8a4bc4"
Sep 17 16:49:03.088884 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:03.088844    2586 scope.go:117] "RemoveContainer" containerID="ae4eb31bd436d24b8bdd9f7f21d34e11ddfdc5e184d74180902879df0e8a4bc4"
Sep 17 16:49:03.089166 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:49:03.089147    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae4eb31bd436d24b8bdd9f7f21d34e11ddfdc5e184d74180902879df0e8a4bc4\": container with ID starting with ae4eb31bd436d24b8bdd9f7f21d34e11ddfdc5e184d74180902879df0e8a4bc4 not found: ID does not exist" containerID="ae4eb31bd436d24b8bdd9f7f21d34e11ddfdc5e184d74180902879df0e8a4bc4"
Sep 17 16:49:03.089270 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:03.089174    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae4eb31bd436d24b8bdd9f7f21d34e11ddfdc5e184d74180902879df0e8a4bc4"} err="failed to get container status \"ae4eb31bd436d24b8bdd9f7f21d34e11ddfdc5e184d74180902879df0e8a4bc4\": rpc error: code = NotFound desc = could not find container \"ae4eb31bd436d24b8bdd9f7f21d34e11ddfdc5e184d74180902879df0e8a4bc4\": container with ID starting with ae4eb31bd436d24b8bdd9f7f21d34e11ddfdc5e184d74180902879df0e8a4bc4 not found: ID does not exist"
Sep 17 16:49:03.100629 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:03.100601    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cc6456699-sfwst"]
Sep 17 16:49:03.104377 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:03.104355    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6cc6456699-sfwst"]
Sep 17 16:49:03.324000 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:03.323966    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc010cbe-909b-4b0a-b293-92627cb28412" path="/var/lib/kubelet/pods/fc010cbe-909b-4b0a-b293-92627cb28412/volumes"
Sep 17 16:49:23.040025 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:23.039934    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-645f9c8fd6-p5dqp"
Sep 17 16:49:23.042729 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:23.042712    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-846cf5d849-hdzwj"
Sep 17 16:49:30.438648 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:30.438610    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-846cf5d849-hdzwj"]
Sep 17 16:49:30.439063 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:30.438829    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve/kserve-controller-manager-846cf5d849-hdzwj" podUID="b198ab34-0f7a-4cfe-8461-6dfc2b682aa3" containerName="manager" containerID="cri-o://1aa380ec96f6455c09be528ca93f45512763accca33a061632979edbe5541ee7" gracePeriod=10
Sep 17 16:49:30.462051 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:30.462022    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-846cf5d849-g7m2q"]
Sep 17 16:49:30.462364 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:30.462352    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc010cbe-909b-4b0a-b293-92627cb28412" containerName="console"
Sep 17 16:49:30.462406 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:30.462367    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc010cbe-909b-4b0a-b293-92627cb28412" containerName="console"
Sep 17 16:49:30.462445 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:30.462423    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc010cbe-909b-4b0a-b293-92627cb28412" containerName="console"
Sep 17 16:49:30.464248 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:30.464233    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-846cf5d849-g7m2q"
Sep 17 16:49:30.474741 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:30.474707    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-846cf5d849-g7m2q"]
Sep 17 16:49:30.513538 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:30.513503    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx76s\" (UniqueName: \"kubernetes.io/projected/40a49f5a-e83c-4853-ae1a-4d3bb6904487-kube-api-access-gx76s\") pod \"kserve-controller-manager-846cf5d849-g7m2q\" (UID: \"40a49f5a-e83c-4853-ae1a-4d3bb6904487\") " pod="kserve/kserve-controller-manager-846cf5d849-g7m2q"
Sep 17 16:49:30.513664 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:30.513561    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/40a49f5a-e83c-4853-ae1a-4d3bb6904487-cert\") pod \"kserve-controller-manager-846cf5d849-g7m2q\" (UID: \"40a49f5a-e83c-4853-ae1a-4d3bb6904487\") " pod="kserve/kserve-controller-manager-846cf5d849-g7m2q"
Sep 17 16:49:30.614486 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:30.614448    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/40a49f5a-e83c-4853-ae1a-4d3bb6904487-cert\") pod \"kserve-controller-manager-846cf5d849-g7m2q\" (UID: \"40a49f5a-e83c-4853-ae1a-4d3bb6904487\") " pod="kserve/kserve-controller-manager-846cf5d849-g7m2q"
Sep 17 16:49:30.614835 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:30.614549    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx76s\" (UniqueName: \"kubernetes.io/projected/40a49f5a-e83c-4853-ae1a-4d3bb6904487-kube-api-access-gx76s\") pod \"kserve-controller-manager-846cf5d849-g7m2q\" (UID: \"40a49f5a-e83c-4853-ae1a-4d3bb6904487\") " pod="kserve/kserve-controller-manager-846cf5d849-g7m2q"
Sep 17 16:49:30.617278 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:30.617247    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/40a49f5a-e83c-4853-ae1a-4d3bb6904487-cert\") pod \"kserve-controller-manager-846cf5d849-g7m2q\" (UID: \"40a49f5a-e83c-4853-ae1a-4d3bb6904487\") " pod="kserve/kserve-controller-manager-846cf5d849-g7m2q"
Sep 17 16:49:30.624518 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:30.624487    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx76s\" (UniqueName: \"kubernetes.io/projected/40a49f5a-e83c-4853-ae1a-4d3bb6904487-kube-api-access-gx76s\") pod \"kserve-controller-manager-846cf5d849-g7m2q\" (UID: \"40a49f5a-e83c-4853-ae1a-4d3bb6904487\") " pod="kserve/kserve-controller-manager-846cf5d849-g7m2q"
Sep 17 16:49:30.678838 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:30.678813    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-846cf5d849-hdzwj"
Sep 17 16:49:30.810695 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:30.810664    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-846cf5d849-g7m2q"
Sep 17 16:49:30.816300 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:30.816281    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpnpq\" (UniqueName: \"kubernetes.io/projected/b198ab34-0f7a-4cfe-8461-6dfc2b682aa3-kube-api-access-qpnpq\") pod \"b198ab34-0f7a-4cfe-8461-6dfc2b682aa3\" (UID: \"b198ab34-0f7a-4cfe-8461-6dfc2b682aa3\") "
Sep 17 16:49:30.816406 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:30.816318    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b198ab34-0f7a-4cfe-8461-6dfc2b682aa3-cert\") pod \"b198ab34-0f7a-4cfe-8461-6dfc2b682aa3\" (UID: \"b198ab34-0f7a-4cfe-8461-6dfc2b682aa3\") "
Sep 17 16:49:30.818621 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:30.818589    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b198ab34-0f7a-4cfe-8461-6dfc2b682aa3-kube-api-access-qpnpq" (OuterVolumeSpecName: "kube-api-access-qpnpq") pod "b198ab34-0f7a-4cfe-8461-6dfc2b682aa3" (UID: "b198ab34-0f7a-4cfe-8461-6dfc2b682aa3"). InnerVolumeSpecName "kube-api-access-qpnpq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:49:30.818621 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:30.818613    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b198ab34-0f7a-4cfe-8461-6dfc2b682aa3-cert" (OuterVolumeSpecName: "cert") pod "b198ab34-0f7a-4cfe-8461-6dfc2b682aa3" (UID: "b198ab34-0f7a-4cfe-8461-6dfc2b682aa3"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:49:30.917469 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:30.917433    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpnpq\" (UniqueName: \"kubernetes.io/projected/b198ab34-0f7a-4cfe-8461-6dfc2b682aa3-kube-api-access-qpnpq\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:49:30.917469 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:30.917464    2586 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b198ab34-0f7a-4cfe-8461-6dfc2b682aa3-cert\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:49:31.079494 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:31.079422    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-846cf5d849-g7m2q"]
Sep 17 16:49:31.082123 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:49:31.082090    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40a49f5a_e83c_4853_ae1a_4d3bb6904487.slice/crio-975fd54b021bfc5ce610aed408a69b001a390376ace43c8bf773f549589077e9 WatchSource:0}: Error finding container 975fd54b021bfc5ce610aed408a69b001a390376ace43c8bf773f549589077e9: Status 404 returned error can't find the container with id 975fd54b021bfc5ce610aed408a69b001a390376ace43c8bf773f549589077e9
Sep 17 16:49:31.197147 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:31.197115    2586 generic.go:356] "Generic (PLEG): container finished" podID="b198ab34-0f7a-4cfe-8461-6dfc2b682aa3" containerID="1aa380ec96f6455c09be528ca93f45512763accca33a061632979edbe5541ee7" exitCode=0
Sep 17 16:49:31.197341 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:31.197160    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-846cf5d849-hdzwj"
Sep 17 16:49:31.197341 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:31.197189    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-846cf5d849-hdzwj" event={"ID":"b198ab34-0f7a-4cfe-8461-6dfc2b682aa3","Type":"ContainerDied","Data":"1aa380ec96f6455c09be528ca93f45512763accca33a061632979edbe5541ee7"}
Sep 17 16:49:31.197341 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:31.197240    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-846cf5d849-hdzwj" event={"ID":"b198ab34-0f7a-4cfe-8461-6dfc2b682aa3","Type":"ContainerDied","Data":"7c8b1b8fd4d118240cf9684c76789762c54d97c2533e36f23124b03ebdd44ad7"}
Sep 17 16:49:31.197341 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:31.197265    2586 scope.go:117] "RemoveContainer" containerID="1aa380ec96f6455c09be528ca93f45512763accca33a061632979edbe5541ee7"
Sep 17 16:49:31.198525 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:31.198504    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-846cf5d849-g7m2q" event={"ID":"40a49f5a-e83c-4853-ae1a-4d3bb6904487","Type":"ContainerStarted","Data":"975fd54b021bfc5ce610aed408a69b001a390376ace43c8bf773f549589077e9"}
Sep 17 16:49:31.209534 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:31.209515    2586 scope.go:117] "RemoveContainer" containerID="1aa380ec96f6455c09be528ca93f45512763accca33a061632979edbe5541ee7"
Sep 17 16:49:31.209816 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:49:31.209796    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1aa380ec96f6455c09be528ca93f45512763accca33a061632979edbe5541ee7\": container with ID starting with 1aa380ec96f6455c09be528ca93f45512763accca33a061632979edbe5541ee7 not found: ID does not exist" containerID="1aa380ec96f6455c09be528ca93f45512763accca33a061632979edbe5541ee7"
Sep 17 16:49:31.209862 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:31.209824    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1aa380ec96f6455c09be528ca93f45512763accca33a061632979edbe5541ee7"} err="failed to get container status \"1aa380ec96f6455c09be528ca93f45512763accca33a061632979edbe5541ee7\": rpc error: code = NotFound desc = could not find container \"1aa380ec96f6455c09be528ca93f45512763accca33a061632979edbe5541ee7\": container with ID starting with 1aa380ec96f6455c09be528ca93f45512763accca33a061632979edbe5541ee7 not found: ID does not exist"
Sep 17 16:49:31.220103 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:31.220072    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-846cf5d849-hdzwj"]
Sep 17 16:49:31.223968 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:31.223948    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-846cf5d849-hdzwj"]
Sep 17 16:49:31.324071 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:31.324028    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b198ab34-0f7a-4cfe-8461-6dfc2b682aa3" path="/var/lib/kubelet/pods/b198ab34-0f7a-4cfe-8461-6dfc2b682aa3/volumes"
Sep 17 16:49:32.205053 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:32.205016    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-846cf5d849-g7m2q" event={"ID":"40a49f5a-e83c-4853-ae1a-4d3bb6904487","Type":"ContainerStarted","Data":"03baaa2adb45d17cb3c7b19e24efbdc74e34870ca1de49956ef8f5162e2b5648"}
Sep 17 16:49:32.205470 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:32.205160    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-846cf5d849-g7m2q"
Sep 17 16:49:32.221129 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:49:32.221069    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-846cf5d849-g7m2q" podStartSLOduration=1.8751143959999999 podStartE2EDuration="2.221047381s" totalImagesPullingTime="345.932985ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:49:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:49:32.220485907 +0000 UTC m=+442.921564584" watchObservedRunningTime="2026-09-17 16:49:32.221047381 +0000 UTC m=+442.922126059"
Sep 17 16:50:03.214835 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:50:03.214800    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-846cf5d849-g7m2q"
Sep 17 16:50:04.497479 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:50:04.497447    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-7b954ffc95-nxrcc"]
Sep 17 16:50:04.497832 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:50:04.497737    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b198ab34-0f7a-4cfe-8461-6dfc2b682aa3" containerName="manager"
Sep 17 16:50:04.497832 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:50:04.497748    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="b198ab34-0f7a-4cfe-8461-6dfc2b682aa3" containerName="manager"
Sep 17 16:50:04.497832 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:50:04.497803    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="b198ab34-0f7a-4cfe-8461-6dfc2b682aa3" containerName="manager"
Sep 17 16:50:04.500645 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:50:04.500627    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-nxrcc"
Sep 17 16:50:04.502995 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:50:04.502965    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\""
Sep 17 16:50:04.503121 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:50:04.503003    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-gnkrd\""
Sep 17 16:50:04.510404 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:50:04.510384    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-nxrcc"]
Sep 17 16:50:04.691850 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:50:04.691816    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff684b6d-4906-4d60-89ae-ae2db177413a-tls-certs\") pod \"model-serving-api-7b954ffc95-nxrcc\" (UID: \"ff684b6d-4906-4d60-89ae-ae2db177413a\") " pod="kserve/model-serving-api-7b954ffc95-nxrcc"
Sep 17 16:50:04.692059 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:50:04.691863    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bdx2\" (UniqueName: \"kubernetes.io/projected/ff684b6d-4906-4d60-89ae-ae2db177413a-kube-api-access-6bdx2\") pod \"model-serving-api-7b954ffc95-nxrcc\" (UID: \"ff684b6d-4906-4d60-89ae-ae2db177413a\") " pod="kserve/model-serving-api-7b954ffc95-nxrcc"
Sep 17 16:50:04.793419 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:50:04.793377    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bdx2\" (UniqueName: \"kubernetes.io/projected/ff684b6d-4906-4d60-89ae-ae2db177413a-kube-api-access-6bdx2\") pod \"model-serving-api-7b954ffc95-nxrcc\" (UID: \"ff684b6d-4906-4d60-89ae-ae2db177413a\") " pod="kserve/model-serving-api-7b954ffc95-nxrcc"
Sep 17 16:50:04.793804 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:50:04.793779    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff684b6d-4906-4d60-89ae-ae2db177413a-tls-certs\") pod \"model-serving-api-7b954ffc95-nxrcc\" (UID: \"ff684b6d-4906-4d60-89ae-ae2db177413a\") " pod="kserve/model-serving-api-7b954ffc95-nxrcc"
Sep 17 16:50:04.798612 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:50:04.798584    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff684b6d-4906-4d60-89ae-ae2db177413a-tls-certs\") pod \"model-serving-api-7b954ffc95-nxrcc\" (UID: \"ff684b6d-4906-4d60-89ae-ae2db177413a\") " pod="kserve/model-serving-api-7b954ffc95-nxrcc"
Sep 17 16:50:04.803381 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:50:04.803352    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bdx2\" (UniqueName: \"kubernetes.io/projected/ff684b6d-4906-4d60-89ae-ae2db177413a-kube-api-access-6bdx2\") pod \"model-serving-api-7b954ffc95-nxrcc\" (UID: \"ff684b6d-4906-4d60-89ae-ae2db177413a\") " pod="kserve/model-serving-api-7b954ffc95-nxrcc"
Sep 17 16:50:04.811008 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:50:04.810985    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-nxrcc"
Sep 17 16:50:05.082242 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:50:05.082039    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-nxrcc"]
Sep 17 16:50:05.084825 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:50:05.084793    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff684b6d_4906_4d60_89ae_ae2db177413a.slice/crio-858d92c80a577c5fc06051fc8ac10272bc5f03cb269972f08db29d7c386dab08 WatchSource:0}: Error finding container 858d92c80a577c5fc06051fc8ac10272bc5f03cb269972f08db29d7c386dab08: Status 404 returned error can't find the container with id 858d92c80a577c5fc06051fc8ac10272bc5f03cb269972f08db29d7c386dab08
Sep 17 16:50:05.346657 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:50:05.346569    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-nxrcc" event={"ID":"ff684b6d-4906-4d60-89ae-ae2db177413a","Type":"ContainerStarted","Data":"858d92c80a577c5fc06051fc8ac10272bc5f03cb269972f08db29d7c386dab08"}
Sep 17 16:50:07.357031 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:50:07.356999    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-nxrcc" event={"ID":"ff684b6d-4906-4d60-89ae-ae2db177413a","Type":"ContainerStarted","Data":"49706ec269a952545ec4c4523b2862cf3fe1444f3be6ca2404abbc8e8385e3cc"}
Sep 17 16:50:07.357482 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:50:07.357084    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-7b954ffc95-nxrcc"
Sep 17 16:50:07.374406 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:50:07.374353    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-7b954ffc95-nxrcc" podStartSLOduration=1.191084886 podStartE2EDuration="3.374337949s" totalImagesPullingTime="2.183253063s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:50:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:50:07.373585659 +0000 UTC m=+478.074664331" watchObservedRunningTime="2026-09-17 16:50:07.374337949 +0000 UTC m=+478.075416642"
Sep 17 16:50:18.366235 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:50:18.366204    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-7b954ffc95-nxrcc"
Sep 17 16:51:51.282172 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:51.282137    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8"]
Sep 17 16:51:51.285525 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:51.285508    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8"
Sep 17 16:51:51.287947 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:51.287911    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Sep 17 16:51:51.289184 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:51.289159    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-rollback-predictor-serving-cert\""
Sep 17 16:51:51.289345 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:51.289163    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Sep 17 16:51:51.289345 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:51.289293    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-tlgx8\""
Sep 17 16:51:51.289345 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:51.289307    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 17 16:51:51.289345 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:51.289166    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-rollback-kube-rbac-proxy-sar-config\""
Sep 17 16:51:51.289345 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:51.289242    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 17 16:51:51.297074 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:51.297054    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8"]
Sep 17 16:51:51.377803 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:51.377765    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c976975e-30dc-4403-bd70-2d75aa18bd40-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-696c5cc6cc-qrqx8\" (UID: \"c976975e-30dc-4403-bd70-2d75aa18bd40\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8"
Sep 17 16:51:51.377803 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:51.377803    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5zj9\" (UniqueName: \"kubernetes.io/projected/c976975e-30dc-4403-bd70-2d75aa18bd40-kube-api-access-s5zj9\") pod \"isvc-canary-rollback-predictor-696c5cc6cc-qrqx8\" (UID: \"c976975e-30dc-4403-bd70-2d75aa18bd40\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8"
Sep 17 16:51:51.378038 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:51.377860    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c976975e-30dc-4403-bd70-2d75aa18bd40-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-696c5cc6cc-qrqx8\" (UID: \"c976975e-30dc-4403-bd70-2d75aa18bd40\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8"
Sep 17 16:51:51.378038 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:51.377937    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c976975e-30dc-4403-bd70-2d75aa18bd40-proxy-tls\") pod \"isvc-canary-rollback-predictor-696c5cc6cc-qrqx8\" (UID: \"c976975e-30dc-4403-bd70-2d75aa18bd40\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8"
Sep 17 16:51:51.378038 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:51.377967    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c976975e-30dc-4403-bd70-2d75aa18bd40-cabundle-cert\") pod \"isvc-canary-rollback-predictor-696c5cc6cc-qrqx8\" (UID: \"c976975e-30dc-4403-bd70-2d75aa18bd40\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8"
Sep 17 16:51:51.479083 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:51.479049    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c976975e-30dc-4403-bd70-2d75aa18bd40-proxy-tls\") pod \"isvc-canary-rollback-predictor-696c5cc6cc-qrqx8\" (UID: \"c976975e-30dc-4403-bd70-2d75aa18bd40\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8"
Sep 17 16:51:51.479249 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:51.479091    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c976975e-30dc-4403-bd70-2d75aa18bd40-cabundle-cert\") pod \"isvc-canary-rollback-predictor-696c5cc6cc-qrqx8\" (UID: \"c976975e-30dc-4403-bd70-2d75aa18bd40\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8"
Sep 17 16:51:51.479249 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:51.479144    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c976975e-30dc-4403-bd70-2d75aa18bd40-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-696c5cc6cc-qrqx8\" (UID: \"c976975e-30dc-4403-bd70-2d75aa18bd40\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8"
Sep 17 16:51:51.479249 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:51.479160    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5zj9\" (UniqueName: \"kubernetes.io/projected/c976975e-30dc-4403-bd70-2d75aa18bd40-kube-api-access-s5zj9\") pod \"isvc-canary-rollback-predictor-696c5cc6cc-qrqx8\" (UID: \"c976975e-30dc-4403-bd70-2d75aa18bd40\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8"
Sep 17 16:51:51.479249 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:51.479189    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c976975e-30dc-4403-bd70-2d75aa18bd40-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-696c5cc6cc-qrqx8\" (UID: \"c976975e-30dc-4403-bd70-2d75aa18bd40\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8"
Sep 17 16:51:51.479671 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:51.479648    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c976975e-30dc-4403-bd70-2d75aa18bd40-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-696c5cc6cc-qrqx8\" (UID: \"c976975e-30dc-4403-bd70-2d75aa18bd40\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8"
Sep 17 16:51:51.480016 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:51.479997    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c976975e-30dc-4403-bd70-2d75aa18bd40-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-696c5cc6cc-qrqx8\" (UID: \"c976975e-30dc-4403-bd70-2d75aa18bd40\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8"
Sep 17 16:51:51.480073 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:51.480022    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c976975e-30dc-4403-bd70-2d75aa18bd40-cabundle-cert\") pod \"isvc-canary-rollback-predictor-696c5cc6cc-qrqx8\" (UID: \"c976975e-30dc-4403-bd70-2d75aa18bd40\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8"
Sep 17 16:51:51.481785 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:51.481764    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c976975e-30dc-4403-bd70-2d75aa18bd40-proxy-tls\") pod \"isvc-canary-rollback-predictor-696c5cc6cc-qrqx8\" (UID: \"c976975e-30dc-4403-bd70-2d75aa18bd40\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8"
Sep 17 16:51:51.487174 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:51.487153    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5zj9\" (UniqueName: \"kubernetes.io/projected/c976975e-30dc-4403-bd70-2d75aa18bd40-kube-api-access-s5zj9\") pod \"isvc-canary-rollback-predictor-696c5cc6cc-qrqx8\" (UID: \"c976975e-30dc-4403-bd70-2d75aa18bd40\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8"
Sep 17 16:51:51.596279 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:51.596188    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8"
Sep 17 16:51:51.868922 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:51.868749    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8"]
Sep 17 16:51:51.871537 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:51:51.871509    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc976975e_30dc_4403_bd70_2d75aa18bd40.slice/crio-9fcda8d8063a52e982ad70ff100c1d39183b2a8ed62467d1f242971373076e6b WatchSource:0}: Error finding container 9fcda8d8063a52e982ad70ff100c1d39183b2a8ed62467d1f242971373076e6b: Status 404 returned error can't find the container with id 9fcda8d8063a52e982ad70ff100c1d39183b2a8ed62467d1f242971373076e6b
Sep 17 16:51:52.793364 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:52.793329    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8" event={"ID":"c976975e-30dc-4403-bd70-2d75aa18bd40","Type":"ContainerStarted","Data":"9fcda8d8063a52e982ad70ff100c1d39183b2a8ed62467d1f242971373076e6b"}
Sep 17 16:51:55.809084 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:55.809046    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8" event={"ID":"c976975e-30dc-4403-bd70-2d75aa18bd40","Type":"ContainerStarted","Data":"301e357628a286e756c83369d668f93eb072661f02d8b367e6b2c4f59450d1ee"}
Sep 17 16:51:59.827676 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:59.827645    2586 generic.go:356] "Generic (PLEG): container finished" podID="c976975e-30dc-4403-bd70-2d75aa18bd40" containerID="301e357628a286e756c83369d668f93eb072661f02d8b367e6b2c4f59450d1ee" exitCode=0
Sep 17 16:51:59.827676 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:51:59.827685    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8" event={"ID":"c976975e-30dc-4403-bd70-2d75aa18bd40","Type":"ContainerDied","Data":"301e357628a286e756c83369d668f93eb072661f02d8b367e6b2c4f59450d1ee"}
Sep 17 16:52:12.905548 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:52:12.905519    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22fdn_68559fca-75ed-4ee8-bedd-57b04fbd3f4d/ovn-acl-logging/0.log"
Sep 17 16:52:12.906014 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:52:12.905829    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22fdn_68559fca-75ed-4ee8-bedd-57b04fbd3f4d/ovn-acl-logging/0.log"
Sep 17 16:52:13.911448 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:52:13.911384    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8" event={"ID":"c976975e-30dc-4403-bd70-2d75aa18bd40","Type":"ContainerStarted","Data":"403c5db06c34676f6e5d902cf57ad476822ac74bd28879c30d2e839a0bd17bf9"}
Sep 17 16:52:15.922760 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:52:15.922726    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8" event={"ID":"c976975e-30dc-4403-bd70-2d75aa18bd40","Type":"ContainerStarted","Data":"37a970a143a243168d55ac8b98459a7f1243d86818c7be0e59644e95ae078067"}
Sep 17 16:52:15.923174 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:52:15.922889    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8"
Sep 17 16:52:15.941589 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:52:15.941537    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8" podStartSLOduration=2.4453444810000002 podStartE2EDuration="24.941521877s" totalImagesPullingTime="18.474793779s" totalInitContainerRuntime="4.021383617s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:51:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:52:15.940066525 +0000 UTC m=+606.641145201" watchObservedRunningTime="2026-09-17 16:52:15.941521877 +0000 UTC m=+606.642600600"
Sep 17 16:52:16.930245 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:52:16.930212    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8"
Sep 17 16:52:16.931623 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:52:16.931590    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8" podUID="c976975e-30dc-4403-bd70-2d75aa18bd40" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Sep 17 16:52:17.933902 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:52:17.933832    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8" podUID="c976975e-30dc-4403-bd70-2d75aa18bd40" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Sep 17 16:52:22.938474 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:52:22.938402    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8"
Sep 17 16:52:22.939075 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:52:22.939045    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8" podUID="c976975e-30dc-4403-bd70-2d75aa18bd40" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Sep 17 16:52:32.939087 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:52:32.939049    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8" podUID="c976975e-30dc-4403-bd70-2d75aa18bd40" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Sep 17 16:52:42.939746 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:52:42.939703    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8" podUID="c976975e-30dc-4403-bd70-2d75aa18bd40" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Sep 17 16:52:52.939117 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:52:52.939077    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8" podUID="c976975e-30dc-4403-bd70-2d75aa18bd40" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Sep 17 16:53:02.939530 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:53:02.939493    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8" podUID="c976975e-30dc-4403-bd70-2d75aa18bd40" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Sep 17 16:53:12.939798 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:53:12.939761    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8" podUID="c976975e-30dc-4403-bd70-2d75aa18bd40" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Sep 17 16:53:22.939860 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:53:22.939815    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8" podUID="c976975e-30dc-4403-bd70-2d75aa18bd40" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Sep 17 16:53:32.940067 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:53:32.940035    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8"
Sep 17 16:55:11.033008 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:11.032976    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8"]
Sep 17 16:55:11.033517 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:11.033225    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8" podUID="c976975e-30dc-4403-bd70-2d75aa18bd40" containerName="kserve-container" containerID="cri-o://403c5db06c34676f6e5d902cf57ad476822ac74bd28879c30d2e839a0bd17bf9" gracePeriod=30
Sep 17 16:55:11.033517 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:11.033289    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8" podUID="c976975e-30dc-4403-bd70-2d75aa18bd40" containerName="kube-rbac-proxy" containerID="cri-o://37a970a143a243168d55ac8b98459a7f1243d86818c7be0e59644e95ae078067" gracePeriod=30
Sep 17 16:55:11.695891 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:11.695833    2586 generic.go:356] "Generic (PLEG): container finished" podID="c976975e-30dc-4403-bd70-2d75aa18bd40" containerID="37a970a143a243168d55ac8b98459a7f1243d86818c7be0e59644e95ae078067" exitCode=2
Sep 17 16:55:11.696053 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:11.695914    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8" event={"ID":"c976975e-30dc-4403-bd70-2d75aa18bd40","Type":"ContainerDied","Data":"37a970a143a243168d55ac8b98459a7f1243d86818c7be0e59644e95ae078067"}
Sep 17 16:55:12.934428 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:12.934388    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8" podUID="c976975e-30dc-4403-bd70-2d75aa18bd40" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.29:8643/healthz\": dial tcp 10.133.0.29:8643: connect: connection refused"
Sep 17 16:55:12.939815 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:12.939786    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8" podUID="c976975e-30dc-4403-bd70-2d75aa18bd40" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Sep 17 16:55:15.678859 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:15.678831    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8"
Sep 17 16:55:15.714331 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:15.714244    2586 generic.go:356] "Generic (PLEG): container finished" podID="c976975e-30dc-4403-bd70-2d75aa18bd40" containerID="403c5db06c34676f6e5d902cf57ad476822ac74bd28879c30d2e839a0bd17bf9" exitCode=0
Sep 17 16:55:15.714331 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:15.714293    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8"
Sep 17 16:55:15.714331 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:15.714308    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8" event={"ID":"c976975e-30dc-4403-bd70-2d75aa18bd40","Type":"ContainerDied","Data":"403c5db06c34676f6e5d902cf57ad476822ac74bd28879c30d2e839a0bd17bf9"}
Sep 17 16:55:15.714525 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:15.714375    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8" event={"ID":"c976975e-30dc-4403-bd70-2d75aa18bd40","Type":"ContainerDied","Data":"9fcda8d8063a52e982ad70ff100c1d39183b2a8ed62467d1f242971373076e6b"}
Sep 17 16:55:15.714525 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:15.714416    2586 scope.go:117] "RemoveContainer" containerID="37a970a143a243168d55ac8b98459a7f1243d86818c7be0e59644e95ae078067"
Sep 17 16:55:15.723104 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:15.723086    2586 scope.go:117] "RemoveContainer" containerID="403c5db06c34676f6e5d902cf57ad476822ac74bd28879c30d2e839a0bd17bf9"
Sep 17 16:55:15.730454 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:15.730438    2586 scope.go:117] "RemoveContainer" containerID="301e357628a286e756c83369d668f93eb072661f02d8b367e6b2c4f59450d1ee"
Sep 17 16:55:15.737471 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:15.737456    2586 scope.go:117] "RemoveContainer" containerID="37a970a143a243168d55ac8b98459a7f1243d86818c7be0e59644e95ae078067"
Sep 17 16:55:15.737735 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:55:15.737712    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37a970a143a243168d55ac8b98459a7f1243d86818c7be0e59644e95ae078067\": container with ID starting with 37a970a143a243168d55ac8b98459a7f1243d86818c7be0e59644e95ae078067 not found: ID does not exist" containerID="37a970a143a243168d55ac8b98459a7f1243d86818c7be0e59644e95ae078067"
Sep 17 16:55:15.737804 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:15.737747    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37a970a143a243168d55ac8b98459a7f1243d86818c7be0e59644e95ae078067"} err="failed to get container status \"37a970a143a243168d55ac8b98459a7f1243d86818c7be0e59644e95ae078067\": rpc error: code = NotFound desc = could not find container \"37a970a143a243168d55ac8b98459a7f1243d86818c7be0e59644e95ae078067\": container with ID starting with 37a970a143a243168d55ac8b98459a7f1243d86818c7be0e59644e95ae078067 not found: ID does not exist"
Sep 17 16:55:15.737804 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:15.737772    2586 scope.go:117] "RemoveContainer" containerID="403c5db06c34676f6e5d902cf57ad476822ac74bd28879c30d2e839a0bd17bf9"
Sep 17 16:55:15.738094 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:55:15.738077    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"403c5db06c34676f6e5d902cf57ad476822ac74bd28879c30d2e839a0bd17bf9\": container with ID starting with 403c5db06c34676f6e5d902cf57ad476822ac74bd28879c30d2e839a0bd17bf9 not found: ID does not exist" containerID="403c5db06c34676f6e5d902cf57ad476822ac74bd28879c30d2e839a0bd17bf9"
Sep 17 16:55:15.738138 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:15.738100    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"403c5db06c34676f6e5d902cf57ad476822ac74bd28879c30d2e839a0bd17bf9"} err="failed to get container status \"403c5db06c34676f6e5d902cf57ad476822ac74bd28879c30d2e839a0bd17bf9\": rpc error: code = NotFound desc = could not find container \"403c5db06c34676f6e5d902cf57ad476822ac74bd28879c30d2e839a0bd17bf9\": container with ID starting with 403c5db06c34676f6e5d902cf57ad476822ac74bd28879c30d2e839a0bd17bf9 not found: ID does not exist"
Sep 17 16:55:15.738138 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:15.738124    2586 scope.go:117] "RemoveContainer" containerID="301e357628a286e756c83369d668f93eb072661f02d8b367e6b2c4f59450d1ee"
Sep 17 16:55:15.738372 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:55:15.738357    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"301e357628a286e756c83369d668f93eb072661f02d8b367e6b2c4f59450d1ee\": container with ID starting with 301e357628a286e756c83369d668f93eb072661f02d8b367e6b2c4f59450d1ee not found: ID does not exist" containerID="301e357628a286e756c83369d668f93eb072661f02d8b367e6b2c4f59450d1ee"
Sep 17 16:55:15.738413 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:15.738377    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"301e357628a286e756c83369d668f93eb072661f02d8b367e6b2c4f59450d1ee"} err="failed to get container status \"301e357628a286e756c83369d668f93eb072661f02d8b367e6b2c4f59450d1ee\": rpc error: code = NotFound desc = could not find container \"301e357628a286e756c83369d668f93eb072661f02d8b367e6b2c4f59450d1ee\": container with ID starting with 301e357628a286e756c83369d668f93eb072661f02d8b367e6b2c4f59450d1ee not found: ID does not exist"
Sep 17 16:55:15.774985 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:15.774961    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5zj9\" (UniqueName: \"kubernetes.io/projected/c976975e-30dc-4403-bd70-2d75aa18bd40-kube-api-access-s5zj9\") pod \"c976975e-30dc-4403-bd70-2d75aa18bd40\" (UID: \"c976975e-30dc-4403-bd70-2d75aa18bd40\") "
Sep 17 16:55:15.775060 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:15.775012    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c976975e-30dc-4403-bd70-2d75aa18bd40-proxy-tls\") pod \"c976975e-30dc-4403-bd70-2d75aa18bd40\" (UID: \"c976975e-30dc-4403-bd70-2d75aa18bd40\") "
Sep 17 16:55:15.775060 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:15.775037    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c976975e-30dc-4403-bd70-2d75aa18bd40-kserve-provision-location\") pod \"c976975e-30dc-4403-bd70-2d75aa18bd40\" (UID: \"c976975e-30dc-4403-bd70-2d75aa18bd40\") "
Sep 17 16:55:15.775060 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:15.775058    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c976975e-30dc-4403-bd70-2d75aa18bd40-kube-rbac-proxy-sar-config\") pod \"c976975e-30dc-4403-bd70-2d75aa18bd40\" (UID: \"c976975e-30dc-4403-bd70-2d75aa18bd40\") "
Sep 17 16:55:15.775162 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:15.775076    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c976975e-30dc-4403-bd70-2d75aa18bd40-cabundle-cert\") pod \"c976975e-30dc-4403-bd70-2d75aa18bd40\" (UID: \"c976975e-30dc-4403-bd70-2d75aa18bd40\") "
Sep 17 16:55:15.775397 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:15.775367    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c976975e-30dc-4403-bd70-2d75aa18bd40-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "c976975e-30dc-4403-bd70-2d75aa18bd40" (UID: "c976975e-30dc-4403-bd70-2d75aa18bd40"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 16:55:15.775480 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:15.775390    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c976975e-30dc-4403-bd70-2d75aa18bd40-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "c976975e-30dc-4403-bd70-2d75aa18bd40" (UID: "c976975e-30dc-4403-bd70-2d75aa18bd40"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:55:15.775524 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:15.775490    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c976975e-30dc-4403-bd70-2d75aa18bd40-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "c976975e-30dc-4403-bd70-2d75aa18bd40" (UID: "c976975e-30dc-4403-bd70-2d75aa18bd40"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:55:15.777222 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:15.777205    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c976975e-30dc-4403-bd70-2d75aa18bd40-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "c976975e-30dc-4403-bd70-2d75aa18bd40" (UID: "c976975e-30dc-4403-bd70-2d75aa18bd40"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:55:15.777350 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:15.777328    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c976975e-30dc-4403-bd70-2d75aa18bd40-kube-api-access-s5zj9" (OuterVolumeSpecName: "kube-api-access-s5zj9") pod "c976975e-30dc-4403-bd70-2d75aa18bd40" (UID: "c976975e-30dc-4403-bd70-2d75aa18bd40"). InnerVolumeSpecName "kube-api-access-s5zj9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:55:15.875706 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:15.875671    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5zj9\" (UniqueName: \"kubernetes.io/projected/c976975e-30dc-4403-bd70-2d75aa18bd40-kube-api-access-s5zj9\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:55:15.875706 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:15.875697    2586 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c976975e-30dc-4403-bd70-2d75aa18bd40-proxy-tls\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:55:15.875706 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:15.875706    2586 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c976975e-30dc-4403-bd70-2d75aa18bd40-kserve-provision-location\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:55:15.875706 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:15.875715    2586 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c976975e-30dc-4403-bd70-2d75aa18bd40-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:55:15.875999 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:15.875723    2586 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c976975e-30dc-4403-bd70-2d75aa18bd40-cabundle-cert\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:55:16.038333 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.038299    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8"]
Sep 17 16:55:16.043735 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.043710    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-696c5cc6cc-qrqx8"]
Sep 17 16:55:16.257476 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.257444    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g"]
Sep 17 16:55:16.257843 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.257825    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c976975e-30dc-4403-bd70-2d75aa18bd40" containerName="kserve-container"
Sep 17 16:55:16.257964 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.257845    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="c976975e-30dc-4403-bd70-2d75aa18bd40" containerName="kserve-container"
Sep 17 16:55:16.257964 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.257855    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c976975e-30dc-4403-bd70-2d75aa18bd40" containerName="kube-rbac-proxy"
Sep 17 16:55:16.257964 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.257863    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="c976975e-30dc-4403-bd70-2d75aa18bd40" containerName="kube-rbac-proxy"
Sep 17 16:55:16.257964 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.257892    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c976975e-30dc-4403-bd70-2d75aa18bd40" containerName="storage-initializer"
Sep 17 16:55:16.257964 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.257902    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="c976975e-30dc-4403-bd70-2d75aa18bd40" containerName="storage-initializer"
Sep 17 16:55:16.258213 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.258000    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="c976975e-30dc-4403-bd70-2d75aa18bd40" containerName="kserve-container"
Sep 17 16:55:16.258213 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.258016    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="c976975e-30dc-4403-bd70-2d75aa18bd40" containerName="kube-rbac-proxy"
Sep 17 16:55:16.262796 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.262774    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g"
Sep 17 16:55:16.265493 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.265437    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-stop-predictor-serving-cert\""
Sep 17 16:55:16.265493 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.265458    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-tlgx8\""
Sep 17 16:55:16.265672 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.265539    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Sep 17 16:55:16.265672 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.265594    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Sep 17 16:55:16.266013 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.265997    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 17 16:55:16.266127 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.266113    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 17 16:55:16.266187 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.266116    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-stop-kube-rbac-proxy-sar-config\""
Sep 17 16:55:16.270480 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.270458    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g"]
Sep 17 16:55:16.279348 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.279291    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgvfg\" (UniqueName: \"kubernetes.io/projected/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3-kube-api-access-lgvfg\") pod \"isvc-canary-stop-predictor-67d6bcd4b5-kmt4g\" (UID: \"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g"
Sep 17 16:55:16.279348 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.279320    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3-cabundle-cert\") pod \"isvc-canary-stop-predictor-67d6bcd4b5-kmt4g\" (UID: \"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g"
Sep 17 16:55:16.279348 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.279348    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3-proxy-tls\") pod \"isvc-canary-stop-predictor-67d6bcd4b5-kmt4g\" (UID: \"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g"
Sep 17 16:55:16.279592 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.279387    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3-kserve-provision-location\") pod \"isvc-canary-stop-predictor-67d6bcd4b5-kmt4g\" (UID: \"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g"
Sep 17 16:55:16.279592 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.279488    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-67d6bcd4b5-kmt4g\" (UID: \"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g"
Sep 17 16:55:16.380062 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.380025    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgvfg\" (UniqueName: \"kubernetes.io/projected/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3-kube-api-access-lgvfg\") pod \"isvc-canary-stop-predictor-67d6bcd4b5-kmt4g\" (UID: \"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g"
Sep 17 16:55:16.380062 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.380058    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3-cabundle-cert\") pod \"isvc-canary-stop-predictor-67d6bcd4b5-kmt4g\" (UID: \"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g"
Sep 17 16:55:16.380351 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.380077    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3-proxy-tls\") pod \"isvc-canary-stop-predictor-67d6bcd4b5-kmt4g\" (UID: \"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g"
Sep 17 16:55:16.380351 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.380091    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3-kserve-provision-location\") pod \"isvc-canary-stop-predictor-67d6bcd4b5-kmt4g\" (UID: \"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g"
Sep 17 16:55:16.380351 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.380142    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-67d6bcd4b5-kmt4g\" (UID: \"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g"
Sep 17 16:55:16.380626 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.380604    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3-kserve-provision-location\") pod \"isvc-canary-stop-predictor-67d6bcd4b5-kmt4g\" (UID: \"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g"
Sep 17 16:55:16.380804 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.380752    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3-cabundle-cert\") pod \"isvc-canary-stop-predictor-67d6bcd4b5-kmt4g\" (UID: \"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g"
Sep 17 16:55:16.381066 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.380825    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-67d6bcd4b5-kmt4g\" (UID: \"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g"
Sep 17 16:55:16.382739 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.382719    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3-proxy-tls\") pod \"isvc-canary-stop-predictor-67d6bcd4b5-kmt4g\" (UID: \"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g"
Sep 17 16:55:16.396240 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.396215    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgvfg\" (UniqueName: \"kubernetes.io/projected/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3-kube-api-access-lgvfg\") pod \"isvc-canary-stop-predictor-67d6bcd4b5-kmt4g\" (UID: \"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g"
Sep 17 16:55:16.574893 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.574788    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g"
Sep 17 16:55:16.848117 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:16.848094    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g"]
Sep 17 16:55:16.850379 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:55:16.850349    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dcd524d_fe6c_46f8_a996_0ce1d2efc7d3.slice/crio-18bbac4f291fd0a81362b6c74a707072b0f21558b90a977529ad65c18a85dd7b WatchSource:0}: Error finding container 18bbac4f291fd0a81362b6c74a707072b0f21558b90a977529ad65c18a85dd7b: Status 404 returned error can't find the container with id 18bbac4f291fd0a81362b6c74a707072b0f21558b90a977529ad65c18a85dd7b
Sep 17 16:55:17.325454 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:17.325410    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c976975e-30dc-4403-bd70-2d75aa18bd40" path="/var/lib/kubelet/pods/c976975e-30dc-4403-bd70-2d75aa18bd40/volumes"
Sep 17 16:55:17.725787 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:17.725703    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g" event={"ID":"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3","Type":"ContainerStarted","Data":"b711dde67ceec453f7b21d3cde4fdc0b1a812493ef1ebaff36449b405268459a"}
Sep 17 16:55:17.725787 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:17.725757    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g" event={"ID":"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3","Type":"ContainerStarted","Data":"18bbac4f291fd0a81362b6c74a707072b0f21558b90a977529ad65c18a85dd7b"}
Sep 17 16:55:21.744085 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:21.744000    2586 generic.go:356] "Generic (PLEG): container finished" podID="3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3" containerID="b711dde67ceec453f7b21d3cde4fdc0b1a812493ef1ebaff36449b405268459a" exitCode=0
Sep 17 16:55:21.744085 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:21.744055    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g" event={"ID":"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3","Type":"ContainerDied","Data":"b711dde67ceec453f7b21d3cde4fdc0b1a812493ef1ebaff36449b405268459a"}
Sep 17 16:55:22.750356 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:22.750321    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g" event={"ID":"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3","Type":"ContainerStarted","Data":"d6c2297bc5250cd500d85ea328857b472727a0e4d9f8d6c197830d92da405463"}
Sep 17 16:55:22.750802 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:22.750382    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g" event={"ID":"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3","Type":"ContainerStarted","Data":"692c93a3f28e00aa18fd28fdb6fb705ebef71b580036f3ad4859a337f902f258"}
Sep 17 16:55:22.750802 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:22.750664    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g"
Sep 17 16:55:22.750802 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:22.750723    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g"
Sep 17 16:55:22.752325 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:22.752296    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g" podUID="3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Sep 17 16:55:22.770504 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:22.770446    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g" podStartSLOduration=2.8937419970000002 podStartE2EDuration="6.770431571s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.876689574s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:55:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:55:22.768741905 +0000 UTC m=+793.469820568" watchObservedRunningTime="2026-09-17 16:55:22.770431571 +0000 UTC m=+793.471510232"
Sep 17 16:55:23.754757 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:23.754716    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g" podUID="3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Sep 17 16:55:26.115095 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:26.115053    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd"]
Sep 17 16:55:26.118088 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:26.118071    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd"
Sep 17 16:55:26.120592 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:26.120568    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-48e77-serving-cert\""
Sep 17 16:55:26.120813 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:26.120794    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-48e77-kube-rbac-proxy-sar-config\""
Sep 17 16:55:26.130015 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:26.129992    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd"]
Sep 17 16:55:26.149450 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:26.149421    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ba4b3d16-0544-415c-936e-2f06880cf582-proxy-tls\") pod \"model-chainer-raw-48e77-77c5845d5-h5tqd\" (UID: \"ba4b3d16-0544-415c-936e-2f06880cf582\") " pod="kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd"
Sep 17 16:55:26.149599 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:26.149467    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba4b3d16-0544-415c-936e-2f06880cf582-openshift-service-ca-bundle\") pod \"model-chainer-raw-48e77-77c5845d5-h5tqd\" (UID: \"ba4b3d16-0544-415c-936e-2f06880cf582\") " pod="kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd"
Sep 17 16:55:26.250762 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:26.250721    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ba4b3d16-0544-415c-936e-2f06880cf582-proxy-tls\") pod \"model-chainer-raw-48e77-77c5845d5-h5tqd\" (UID: \"ba4b3d16-0544-415c-936e-2f06880cf582\") " pod="kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd"
Sep 17 16:55:26.250964 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:26.250780    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba4b3d16-0544-415c-936e-2f06880cf582-openshift-service-ca-bundle\") pod \"model-chainer-raw-48e77-77c5845d5-h5tqd\" (UID: \"ba4b3d16-0544-415c-936e-2f06880cf582\") " pod="kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd"
Sep 17 16:55:26.251584 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:26.251548    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba4b3d16-0544-415c-936e-2f06880cf582-openshift-service-ca-bundle\") pod \"model-chainer-raw-48e77-77c5845d5-h5tqd\" (UID: \"ba4b3d16-0544-415c-936e-2f06880cf582\") " pod="kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd"
Sep 17 16:55:26.253379 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:26.253356    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ba4b3d16-0544-415c-936e-2f06880cf582-proxy-tls\") pod \"model-chainer-raw-48e77-77c5845d5-h5tqd\" (UID: \"ba4b3d16-0544-415c-936e-2f06880cf582\") " pod="kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd"
Sep 17 16:55:26.428334 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:26.428257    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd"
Sep 17 16:55:26.701032 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:26.701007    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd"]
Sep 17 16:55:26.703526 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:55:26.703502    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba4b3d16_0544_415c_936e_2f06880cf582.slice/crio-5dd84502623f185b0337b1fd79e763421544d225339d04a60d9be9d74b50eea1 WatchSource:0}: Error finding container 5dd84502623f185b0337b1fd79e763421544d225339d04a60d9be9d74b50eea1: Status 404 returned error can't find the container with id 5dd84502623f185b0337b1fd79e763421544d225339d04a60d9be9d74b50eea1
Sep 17 16:55:26.706029 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:26.706011    2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 16:55:26.767792 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:26.767751    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd" event={"ID":"ba4b3d16-0544-415c-936e-2f06880cf582","Type":"ContainerStarted","Data":"5dd84502623f185b0337b1fd79e763421544d225339d04a60d9be9d74b50eea1"}
Sep 17 16:55:28.760390 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:28.760361    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g"
Sep 17 16:55:28.761160 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:28.761128    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g" podUID="3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Sep 17 16:55:29.783562 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:29.783522    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd" event={"ID":"ba4b3d16-0544-415c-936e-2f06880cf582","Type":"ContainerStarted","Data":"990675afdf74a9452598b400b5727784d9838f2bb1468b0bd84edf998cf14df9"}
Sep 17 16:55:29.784069 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:29.783594    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd"
Sep 17 16:55:29.801077 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:29.801029    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd" podStartSLOduration=1.581116178 podStartE2EDuration="3.801012956s" totalImagesPullingTime="2.219896778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:55:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:55:29.80016041 +0000 UTC m=+800.501239073" watchObservedRunningTime="2026-09-17 16:55:29.801012956 +0000 UTC m=+800.502091618"
Sep 17 16:55:35.793629 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:35.793598    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd"
Sep 17 16:55:36.194856 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:36.194775    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd"]
Sep 17 16:55:36.195164 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:36.195138    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd" podUID="ba4b3d16-0544-415c-936e-2f06880cf582" containerName="model-chainer-raw-48e77" containerID="cri-o://990675afdf74a9452598b400b5727784d9838f2bb1468b0bd84edf998cf14df9" gracePeriod=30
Sep 17 16:55:38.761667 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:38.761630    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g" podUID="3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Sep 17 16:55:40.791820 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:40.791779    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd" podUID="ba4b3d16-0544-415c-936e-2f06880cf582" containerName="model-chainer-raw-48e77" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:55:45.792367 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:45.792322    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd" podUID="ba4b3d16-0544-415c-936e-2f06880cf582" containerName="model-chainer-raw-48e77" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:55:48.761573 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:48.761535    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g" podUID="3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Sep 17 16:55:50.791837 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:50.791798    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd" podUID="ba4b3d16-0544-415c-936e-2f06880cf582" containerName="model-chainer-raw-48e77" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:55:50.792291 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:50.791911    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd"
Sep 17 16:55:55.791589 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:55.791547    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd" podUID="ba4b3d16-0544-415c-936e-2f06880cf582" containerName="model-chainer-raw-48e77" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:55:58.761937 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:55:58.761896    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g" podUID="3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Sep 17 16:56:00.792104 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:00.792062    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd" podUID="ba4b3d16-0544-415c-936e-2f06880cf582" containerName="model-chainer-raw-48e77" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:56:05.792492 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:05.792452    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd" podUID="ba4b3d16-0544-415c-936e-2f06880cf582" containerName="model-chainer-raw-48e77" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:56:06.835854 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:06.835830    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd"
Sep 17 16:56:06.956617 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:06.956576    2586 generic.go:356] "Generic (PLEG): container finished" podID="ba4b3d16-0544-415c-936e-2f06880cf582" containerID="990675afdf74a9452598b400b5727784d9838f2bb1468b0bd84edf998cf14df9" exitCode=0
Sep 17 16:56:06.956782 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:06.956625    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd"
Sep 17 16:56:06.956782 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:06.956664    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd" event={"ID":"ba4b3d16-0544-415c-936e-2f06880cf582","Type":"ContainerDied","Data":"990675afdf74a9452598b400b5727784d9838f2bb1468b0bd84edf998cf14df9"}
Sep 17 16:56:06.956782 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:06.956724    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd" event={"ID":"ba4b3d16-0544-415c-936e-2f06880cf582","Type":"ContainerDied","Data":"5dd84502623f185b0337b1fd79e763421544d225339d04a60d9be9d74b50eea1"}
Sep 17 16:56:06.956782 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:06.956747    2586 scope.go:117] "RemoveContainer" containerID="990675afdf74a9452598b400b5727784d9838f2bb1468b0bd84edf998cf14df9"
Sep 17 16:56:06.965229 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:06.965211    2586 scope.go:117] "RemoveContainer" containerID="990675afdf74a9452598b400b5727784d9838f2bb1468b0bd84edf998cf14df9"
Sep 17 16:56:06.965507 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:56:06.965485    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"990675afdf74a9452598b400b5727784d9838f2bb1468b0bd84edf998cf14df9\": container with ID starting with 990675afdf74a9452598b400b5727784d9838f2bb1468b0bd84edf998cf14df9 not found: ID does not exist" containerID="990675afdf74a9452598b400b5727784d9838f2bb1468b0bd84edf998cf14df9"
Sep 17 16:56:06.965574 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:06.965516    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"990675afdf74a9452598b400b5727784d9838f2bb1468b0bd84edf998cf14df9"} err="failed to get container status \"990675afdf74a9452598b400b5727784d9838f2bb1468b0bd84edf998cf14df9\": rpc error: code = NotFound desc = could not find container \"990675afdf74a9452598b400b5727784d9838f2bb1468b0bd84edf998cf14df9\": container with ID starting with 990675afdf74a9452598b400b5727784d9838f2bb1468b0bd84edf998cf14df9 not found: ID does not exist"
Sep 17 16:56:07.003897 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:07.003833    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba4b3d16-0544-415c-936e-2f06880cf582-openshift-service-ca-bundle\") pod \"ba4b3d16-0544-415c-936e-2f06880cf582\" (UID: \"ba4b3d16-0544-415c-936e-2f06880cf582\") "
Sep 17 16:56:07.004054 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:07.003919    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ba4b3d16-0544-415c-936e-2f06880cf582-proxy-tls\") pod \"ba4b3d16-0544-415c-936e-2f06880cf582\" (UID: \"ba4b3d16-0544-415c-936e-2f06880cf582\") "
Sep 17 16:56:07.004259 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:07.004229    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba4b3d16-0544-415c-936e-2f06880cf582-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "ba4b3d16-0544-415c-936e-2f06880cf582" (UID: "ba4b3d16-0544-415c-936e-2f06880cf582"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:56:07.006133 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:07.006111    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba4b3d16-0544-415c-936e-2f06880cf582-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ba4b3d16-0544-415c-936e-2f06880cf582" (UID: "ba4b3d16-0544-415c-936e-2f06880cf582"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:56:07.104894 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:07.104845    2586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba4b3d16-0544-415c-936e-2f06880cf582-openshift-service-ca-bundle\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:56:07.104894 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:07.104898    2586 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ba4b3d16-0544-415c-936e-2f06880cf582-proxy-tls\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:56:07.276372 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:07.276345    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd"]
Sep 17 16:56:07.280074 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:07.280044    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-48e77-77c5845d5-h5tqd"]
Sep 17 16:56:07.323724 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:07.323698    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba4b3d16-0544-415c-936e-2f06880cf582" path="/var/lib/kubelet/pods/ba4b3d16-0544-415c-936e-2f06880cf582/volumes"
Sep 17 16:56:08.761479 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:08.761435    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g" podUID="3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Sep 17 16:56:18.761520 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:18.761482    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g" podUID="3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Sep 17 16:56:28.761490 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:28.761450    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g" podUID="3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Sep 17 16:56:38.277211 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:38.277178    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g"]
Sep 17 16:56:38.277625 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:38.277482    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g" podUID="3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3" containerName="kserve-container" containerID="cri-o://692c93a3f28e00aa18fd28fdb6fb705ebef71b580036f3ad4859a337f902f258" gracePeriod=30
Sep 17 16:56:38.277625 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:38.277515    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g" podUID="3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3" containerName="kube-rbac-proxy" containerID="cri-o://d6c2297bc5250cd500d85ea328857b472727a0e4d9f8d6c197830d92da405463" gracePeriod=30
Sep 17 16:56:38.278690 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:38.278671    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g"
Sep 17 16:56:38.755334 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:38.755245    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g" podUID="3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.30:8643/healthz\": dial tcp 10.133.0.30:8643: connect: connection refused"
Sep 17 16:56:39.097693 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:39.097654    2586 generic.go:356] "Generic (PLEG): container finished" podID="3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3" containerID="d6c2297bc5250cd500d85ea328857b472727a0e4d9f8d6c197830d92da405463" exitCode=2
Sep 17 16:56:39.097858 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:39.097722    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g" event={"ID":"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3","Type":"ContainerDied","Data":"d6c2297bc5250cd500d85ea328857b472727a0e4d9f8d6c197830d92da405463"}
Sep 17 16:56:42.932369 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:42.932345    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g"
Sep 17 16:56:42.986796 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:42.986760    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3-kube-rbac-proxy-sar-config\") pod \"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3\" (UID: \"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3\") "
Sep 17 16:56:42.986984 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:42.986901    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3-proxy-tls\") pod \"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3\" (UID: \"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3\") "
Sep 17 16:56:42.986984 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:42.986945    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3-kserve-provision-location\") pod \"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3\" (UID: \"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3\") "
Sep 17 16:56:42.986984 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:42.986973    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lgvfg\" (UniqueName: \"kubernetes.io/projected/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3-kube-api-access-lgvfg\") pod \"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3\" (UID: \"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3\") "
Sep 17 16:56:42.987143 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:42.987002    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3-cabundle-cert\") pod \"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3\" (UID: \"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3\") "
Sep 17 16:56:42.987321 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:42.987288    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3" (UID: "3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 16:56:42.987400 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:42.987265    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3" (UID: "3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:56:42.987442 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:42.987428    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3" (UID: "3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:56:42.989289 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:42.989261    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3" (UID: "3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:56:42.989391 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:42.989323    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3-kube-api-access-lgvfg" (OuterVolumeSpecName: "kube-api-access-lgvfg") pod "3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3" (UID: "3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3"). InnerVolumeSpecName "kube-api-access-lgvfg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:56:43.088328 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:43.088298    2586 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3-kserve-provision-location\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:56:43.088328 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:43.088328    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgvfg\" (UniqueName: \"kubernetes.io/projected/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3-kube-api-access-lgvfg\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:56:43.088517 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:43.088338    2586 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3-cabundle-cert\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:56:43.088517 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:43.088346    2586 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:56:43.088517 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:43.088361    2586 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3-proxy-tls\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:56:43.115658 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:43.115625    2586 generic.go:356] "Generic (PLEG): container finished" podID="3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3" containerID="692c93a3f28e00aa18fd28fdb6fb705ebef71b580036f3ad4859a337f902f258" exitCode=0
Sep 17 16:56:43.115833 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:43.115671    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g" event={"ID":"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3","Type":"ContainerDied","Data":"692c93a3f28e00aa18fd28fdb6fb705ebef71b580036f3ad4859a337f902f258"}
Sep 17 16:56:43.115833 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:43.115736    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g" event={"ID":"3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3","Type":"ContainerDied","Data":"18bbac4f291fd0a81362b6c74a707072b0f21558b90a977529ad65c18a85dd7b"}
Sep 17 16:56:43.115833 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:43.115684    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g"
Sep 17 16:56:43.115981 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:43.115763    2586 scope.go:117] "RemoveContainer" containerID="d6c2297bc5250cd500d85ea328857b472727a0e4d9f8d6c197830d92da405463"
Sep 17 16:56:43.124383 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:43.124364    2586 scope.go:117] "RemoveContainer" containerID="692c93a3f28e00aa18fd28fdb6fb705ebef71b580036f3ad4859a337f902f258"
Sep 17 16:56:43.131804 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:43.131786    2586 scope.go:117] "RemoveContainer" containerID="b711dde67ceec453f7b21d3cde4fdc0b1a812493ef1ebaff36449b405268459a"
Sep 17 16:56:43.136859 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:43.136840    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g"]
Sep 17 16:56:43.139962 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:43.139943    2586 scope.go:117] "RemoveContainer" containerID="d6c2297bc5250cd500d85ea328857b472727a0e4d9f8d6c197830d92da405463"
Sep 17 16:56:43.140164 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:43.140148    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-predictor-67d6bcd4b5-kmt4g"]
Sep 17 16:56:43.140247 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:56:43.140231    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6c2297bc5250cd500d85ea328857b472727a0e4d9f8d6c197830d92da405463\": container with ID starting with d6c2297bc5250cd500d85ea328857b472727a0e4d9f8d6c197830d92da405463 not found: ID does not exist" containerID="d6c2297bc5250cd500d85ea328857b472727a0e4d9f8d6c197830d92da405463"
Sep 17 16:56:43.140289 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:43.140256    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6c2297bc5250cd500d85ea328857b472727a0e4d9f8d6c197830d92da405463"} err="failed to get container status \"d6c2297bc5250cd500d85ea328857b472727a0e4d9f8d6c197830d92da405463\": rpc error: code = NotFound desc = could not find container \"d6c2297bc5250cd500d85ea328857b472727a0e4d9f8d6c197830d92da405463\": container with ID starting with d6c2297bc5250cd500d85ea328857b472727a0e4d9f8d6c197830d92da405463 not found: ID does not exist"
Sep 17 16:56:43.140289 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:43.140272    2586 scope.go:117] "RemoveContainer" containerID="692c93a3f28e00aa18fd28fdb6fb705ebef71b580036f3ad4859a337f902f258"
Sep 17 16:56:43.140517 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:56:43.140503    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"692c93a3f28e00aa18fd28fdb6fb705ebef71b580036f3ad4859a337f902f258\": container with ID starting with 692c93a3f28e00aa18fd28fdb6fb705ebef71b580036f3ad4859a337f902f258 not found: ID does not exist" containerID="692c93a3f28e00aa18fd28fdb6fb705ebef71b580036f3ad4859a337f902f258"
Sep 17 16:56:43.140554 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:43.140520    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"692c93a3f28e00aa18fd28fdb6fb705ebef71b580036f3ad4859a337f902f258"} err="failed to get container status \"692c93a3f28e00aa18fd28fdb6fb705ebef71b580036f3ad4859a337f902f258\": rpc error: code = NotFound desc = could not find container \"692c93a3f28e00aa18fd28fdb6fb705ebef71b580036f3ad4859a337f902f258\": container with ID starting with 692c93a3f28e00aa18fd28fdb6fb705ebef71b580036f3ad4859a337f902f258 not found: ID does not exist"
Sep 17 16:56:43.140554 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:43.140532    2586 scope.go:117] "RemoveContainer" containerID="b711dde67ceec453f7b21d3cde4fdc0b1a812493ef1ebaff36449b405268459a"
Sep 17 16:56:43.140772 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:56:43.140757    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b711dde67ceec453f7b21d3cde4fdc0b1a812493ef1ebaff36449b405268459a\": container with ID starting with b711dde67ceec453f7b21d3cde4fdc0b1a812493ef1ebaff36449b405268459a not found: ID does not exist" containerID="b711dde67ceec453f7b21d3cde4fdc0b1a812493ef1ebaff36449b405268459a"
Sep 17 16:56:43.140807 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:43.140776    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b711dde67ceec453f7b21d3cde4fdc0b1a812493ef1ebaff36449b405268459a"} err="failed to get container status \"b711dde67ceec453f7b21d3cde4fdc0b1a812493ef1ebaff36449b405268459a\": rpc error: code = NotFound desc = could not find container \"b711dde67ceec453f7b21d3cde4fdc0b1a812493ef1ebaff36449b405268459a\": container with ID starting with b711dde67ceec453f7b21d3cde4fdc0b1a812493ef1ebaff36449b405268459a not found: ID does not exist"
Sep 17 16:56:43.324084 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:56:43.324042    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3" path="/var/lib/kubelet/pods/3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3/volumes"
Sep 17 16:57:12.935006 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:12.934323    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22fdn_68559fca-75ed-4ee8-bedd-57b04fbd3f4d/ovn-acl-logging/0.log"
Sep 17 16:57:12.935006 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:12.934572    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22fdn_68559fca-75ed-4ee8-bedd-57b04fbd3f4d/ovn-acl-logging/0.log"
Sep 17 16:57:16.417281 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:16.417247    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss"]
Sep 17 16:57:16.417635 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:16.417547    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3" containerName="storage-initializer"
Sep 17 16:57:16.417635 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:16.417558    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3" containerName="storage-initializer"
Sep 17 16:57:16.417635 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:16.417568    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba4b3d16-0544-415c-936e-2f06880cf582" containerName="model-chainer-raw-48e77"
Sep 17 16:57:16.417635 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:16.417573    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba4b3d16-0544-415c-936e-2f06880cf582" containerName="model-chainer-raw-48e77"
Sep 17 16:57:16.417635 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:16.417583    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3" containerName="kserve-container"
Sep 17 16:57:16.417635 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:16.417589    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3" containerName="kserve-container"
Sep 17 16:57:16.417635 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:16.417606    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3" containerName="kube-rbac-proxy"
Sep 17 16:57:16.417635 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:16.417614    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3" containerName="kube-rbac-proxy"
Sep 17 16:57:16.417937 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:16.417662    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3" containerName="kserve-container"
Sep 17 16:57:16.417937 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:16.417671    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="3dcd524d-fe6c-46f8-a996-0ce1d2efc7d3" containerName="kube-rbac-proxy"
Sep 17 16:57:16.417937 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:16.417677    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba4b3d16-0544-415c-936e-2f06880cf582" containerName="model-chainer-raw-48e77"
Sep 17 16:57:16.420786 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:16.420771    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss"
Sep 17 16:57:16.423222 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:16.423200    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-hpa-48552-serving-cert\""
Sep 17 16:57:16.423352 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:16.423315    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-hpa-48552-kube-rbac-proxy-sar-config\""
Sep 17 16:57:16.423587 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:16.423572    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-tlgx8\""
Sep 17 16:57:16.424384 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:16.424369    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 17 16:57:16.428811 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:16.428792    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss"]
Sep 17 16:57:16.563537 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:16.563495    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/62079efe-2c15-4e14-a520-dbce8f8b0a50-proxy-tls\") pod \"model-chainer-raw-hpa-48552-86d85b86d-lrnss\" (UID: \"62079efe-2c15-4e14-a520-dbce8f8b0a50\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss"
Sep 17 16:57:16.563708 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:16.563543    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62079efe-2c15-4e14-a520-dbce8f8b0a50-openshift-service-ca-bundle\") pod \"model-chainer-raw-hpa-48552-86d85b86d-lrnss\" (UID: \"62079efe-2c15-4e14-a520-dbce8f8b0a50\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss"
Sep 17 16:57:16.664856 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:16.664809    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/62079efe-2c15-4e14-a520-dbce8f8b0a50-proxy-tls\") pod \"model-chainer-raw-hpa-48552-86d85b86d-lrnss\" (UID: \"62079efe-2c15-4e14-a520-dbce8f8b0a50\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss"
Sep 17 16:57:16.664856 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:16.664859    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62079efe-2c15-4e14-a520-dbce8f8b0a50-openshift-service-ca-bundle\") pod \"model-chainer-raw-hpa-48552-86d85b86d-lrnss\" (UID: \"62079efe-2c15-4e14-a520-dbce8f8b0a50\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss"
Sep 17 16:57:16.665131 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:57:16.664989    2586 secret.go:189] Couldn't get secret kserve-ci-e2e-test/model-chainer-raw-hpa-48552-serving-cert: secret "model-chainer-raw-hpa-48552-serving-cert" not found
Sep 17 16:57:16.665131 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:57:16.665063    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62079efe-2c15-4e14-a520-dbce8f8b0a50-proxy-tls podName:62079efe-2c15-4e14-a520-dbce8f8b0a50 nodeName:}" failed. No retries permitted until 2026-09-17 16:57:17.165045136 +0000 UTC m=+907.866123776 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/62079efe-2c15-4e14-a520-dbce8f8b0a50-proxy-tls") pod "model-chainer-raw-hpa-48552-86d85b86d-lrnss" (UID: "62079efe-2c15-4e14-a520-dbce8f8b0a50") : secret "model-chainer-raw-hpa-48552-serving-cert" not found
Sep 17 16:57:16.665569 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:16.665544    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62079efe-2c15-4e14-a520-dbce8f8b0a50-openshift-service-ca-bundle\") pod \"model-chainer-raw-hpa-48552-86d85b86d-lrnss\" (UID: \"62079efe-2c15-4e14-a520-dbce8f8b0a50\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss"
Sep 17 16:57:17.169716 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:17.169656    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/62079efe-2c15-4e14-a520-dbce8f8b0a50-proxy-tls\") pod \"model-chainer-raw-hpa-48552-86d85b86d-lrnss\" (UID: \"62079efe-2c15-4e14-a520-dbce8f8b0a50\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss"
Sep 17 16:57:17.172266 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:17.172237    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/62079efe-2c15-4e14-a520-dbce8f8b0a50-proxy-tls\") pod \"model-chainer-raw-hpa-48552-86d85b86d-lrnss\" (UID: \"62079efe-2c15-4e14-a520-dbce8f8b0a50\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss"
Sep 17 16:57:17.331741 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:17.331707    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss"
Sep 17 16:57:17.602137 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:17.602104    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss"]
Sep 17 16:57:17.605219 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:57:17.605187    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62079efe_2c15_4e14_a520_dbce8f8b0a50.slice/crio-51256d6e67995f0e4bef67125f882f9448130fe3e6dfbe1647fee453e619ca28 WatchSource:0}: Error finding container 51256d6e67995f0e4bef67125f882f9448130fe3e6dfbe1647fee453e619ca28: Status 404 returned error can't find the container with id 51256d6e67995f0e4bef67125f882f9448130fe3e6dfbe1647fee453e619ca28
Sep 17 16:57:18.261244 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:18.261208    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss" event={"ID":"62079efe-2c15-4e14-a520-dbce8f8b0a50","Type":"ContainerStarted","Data":"1a6a367ded65a72bea206890e562dc99679afc76d744e5ad2ebc3c1605f8ea9d"}
Sep 17 16:57:18.261447 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:18.261259    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss" event={"ID":"62079efe-2c15-4e14-a520-dbce8f8b0a50","Type":"ContainerStarted","Data":"51256d6e67995f0e4bef67125f882f9448130fe3e6dfbe1647fee453e619ca28"}
Sep 17 16:57:18.261447 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:18.261302    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss"
Sep 17 16:57:18.276358 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:18.276307    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss" podStartSLOduration=2.2762941420000002 podStartE2EDuration="2.276294142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:57:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:57:18.27621309 +0000 UTC m=+908.977291752" watchObservedRunningTime="2026-09-17 16:57:18.276294142 +0000 UTC m=+908.977372804"
Sep 17 16:57:24.270385 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:24.270350    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss"
Sep 17 16:57:26.474367 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:26.474331    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss"]
Sep 17 16:57:26.474813 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:26.474585    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss" podUID="62079efe-2c15-4e14-a520-dbce8f8b0a50" containerName="model-chainer-raw-hpa-48552" containerID="cri-o://1a6a367ded65a72bea206890e562dc99679afc76d744e5ad2ebc3c1605f8ea9d" gracePeriod=30
Sep 17 16:57:29.268783 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:29.268730    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss" podUID="62079efe-2c15-4e14-a520-dbce8f8b0a50" containerName="model-chainer-raw-hpa-48552" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:57:34.269285 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:34.269245    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss" podUID="62079efe-2c15-4e14-a520-dbce8f8b0a50" containerName="model-chainer-raw-hpa-48552" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:57:39.268982 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:39.268935    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss" podUID="62079efe-2c15-4e14-a520-dbce8f8b0a50" containerName="model-chainer-raw-hpa-48552" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:57:39.269399 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:39.269065    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss"
Sep 17 16:57:44.268992 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:44.268945    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss" podUID="62079efe-2c15-4e14-a520-dbce8f8b0a50" containerName="model-chainer-raw-hpa-48552" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:57:49.269164 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:49.269121    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss" podUID="62079efe-2c15-4e14-a520-dbce8f8b0a50" containerName="model-chainer-raw-hpa-48552" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:57:54.269516 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:54.269476    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss" podUID="62079efe-2c15-4e14-a520-dbce8f8b0a50" containerName="model-chainer-raw-hpa-48552" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 16:57:56.615798 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:56.615768    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss"
Sep 17 16:57:56.680485 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:56.680453    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/62079efe-2c15-4e14-a520-dbce8f8b0a50-proxy-tls\") pod \"62079efe-2c15-4e14-a520-dbce8f8b0a50\" (UID: \"62079efe-2c15-4e14-a520-dbce8f8b0a50\") "
Sep 17 16:57:56.680669 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:56.680555    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62079efe-2c15-4e14-a520-dbce8f8b0a50-openshift-service-ca-bundle\") pod \"62079efe-2c15-4e14-a520-dbce8f8b0a50\" (UID: \"62079efe-2c15-4e14-a520-dbce8f8b0a50\") "
Sep 17 16:57:56.680953 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:56.680930    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62079efe-2c15-4e14-a520-dbce8f8b0a50-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "62079efe-2c15-4e14-a520-dbce8f8b0a50" (UID: "62079efe-2c15-4e14-a520-dbce8f8b0a50"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:57:56.682705 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:56.682675    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62079efe-2c15-4e14-a520-dbce8f8b0a50-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "62079efe-2c15-4e14-a520-dbce8f8b0a50" (UID: "62079efe-2c15-4e14-a520-dbce8f8b0a50"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:57:56.781944 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:56.781908    2586 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/62079efe-2c15-4e14-a520-dbce8f8b0a50-proxy-tls\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:57:56.781944 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:56.781940    2586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62079efe-2c15-4e14-a520-dbce8f8b0a50-openshift-service-ca-bundle\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:57:57.417508 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:57.417473    2586 generic.go:356] "Generic (PLEG): container finished" podID="62079efe-2c15-4e14-a520-dbce8f8b0a50" containerID="1a6a367ded65a72bea206890e562dc99679afc76d744e5ad2ebc3c1605f8ea9d" exitCode=0
Sep 17 16:57:57.417665 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:57.417515    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss"
Sep 17 16:57:57.417665 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:57.417555    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss" event={"ID":"62079efe-2c15-4e14-a520-dbce8f8b0a50","Type":"ContainerDied","Data":"1a6a367ded65a72bea206890e562dc99679afc76d744e5ad2ebc3c1605f8ea9d"}
Sep 17 16:57:57.417665 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:57.417604    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss" event={"ID":"62079efe-2c15-4e14-a520-dbce8f8b0a50","Type":"ContainerDied","Data":"51256d6e67995f0e4bef67125f882f9448130fe3e6dfbe1647fee453e619ca28"}
Sep 17 16:57:57.417665 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:57.417626    2586 scope.go:117] "RemoveContainer" containerID="1a6a367ded65a72bea206890e562dc99679afc76d744e5ad2ebc3c1605f8ea9d"
Sep 17 16:57:57.425853 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:57.425829    2586 scope.go:117] "RemoveContainer" containerID="1a6a367ded65a72bea206890e562dc99679afc76d744e5ad2ebc3c1605f8ea9d"
Sep 17 16:57:57.426162 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:57:57.426142    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a6a367ded65a72bea206890e562dc99679afc76d744e5ad2ebc3c1605f8ea9d\": container with ID starting with 1a6a367ded65a72bea206890e562dc99679afc76d744e5ad2ebc3c1605f8ea9d not found: ID does not exist" containerID="1a6a367ded65a72bea206890e562dc99679afc76d744e5ad2ebc3c1605f8ea9d"
Sep 17 16:57:57.426224 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:57.426170    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a6a367ded65a72bea206890e562dc99679afc76d744e5ad2ebc3c1605f8ea9d"} err="failed to get container status \"1a6a367ded65a72bea206890e562dc99679afc76d744e5ad2ebc3c1605f8ea9d\": rpc error: code = NotFound desc = could not find container \"1a6a367ded65a72bea206890e562dc99679afc76d744e5ad2ebc3c1605f8ea9d\": container with ID starting with 1a6a367ded65a72bea206890e562dc99679afc76d744e5ad2ebc3c1605f8ea9d not found: ID does not exist"
Sep 17 16:57:57.434018 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:57.433993    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss"]
Sep 17 16:57:57.437930 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:57.437909    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-48552-86d85b86d-lrnss"]
Sep 17 16:57:59.324629 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:57:59.324596    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62079efe-2c15-4e14-a520-dbce8f8b0a50" path="/var/lib/kubelet/pods/62079efe-2c15-4e14-a520-dbce8f8b0a50/volumes"
Sep 17 16:58:41.621852 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:41.621805    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv"]
Sep 17 16:58:41.622425 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:41.622166    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62079efe-2c15-4e14-a520-dbce8f8b0a50" containerName="model-chainer-raw-hpa-48552"
Sep 17 16:58:41.622425 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:41.622184    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="62079efe-2c15-4e14-a520-dbce8f8b0a50" containerName="model-chainer-raw-hpa-48552"
Sep 17 16:58:41.622425 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:41.622259    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="62079efe-2c15-4e14-a520-dbce8f8b0a50" containerName="model-chainer-raw-hpa-48552"
Sep 17 16:58:41.625181 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:41.625165    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv"
Sep 17 16:58:41.627518 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:41.627495    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"invalid-s3-sa-42374c-dockercfg-t58jk\""
Sep 17 16:58:41.627765 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:41.627744    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-secondary-42374c-kube-rbac-proxy-sar-config\""
Sep 17 16:58:41.627858 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:41.627774    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Sep 17 16:58:41.627858 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:41.627819    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"invalid-s3-secret-42374c\""
Sep 17 16:58:41.628606 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:41.628585    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-secondary-42374c-predictor-serving-cert\""
Sep 17 16:58:41.628688 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:41.628588    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 17 16:58:41.628688 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:41.628613    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 17 16:58:41.635038 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:41.635012    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv"]
Sep 17 16:58:41.650900 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:41.650847    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gm8t5\" (UniqueName: \"kubernetes.io/projected/ed2282ce-e0bf-48ce-af2a-d9c497d56398-kube-api-access-gm8t5\") pod \"isvc-secondary-42374c-predictor-86544c997d-ch6dv\" (UID: \"ed2282ce-e0bf-48ce-af2a-d9c497d56398\") " pod="kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv"
Sep 17 16:58:41.651049 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:41.651021    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ed2282ce-e0bf-48ce-af2a-d9c497d56398-proxy-tls\") pod \"isvc-secondary-42374c-predictor-86544c997d-ch6dv\" (UID: \"ed2282ce-e0bf-48ce-af2a-d9c497d56398\") " pod="kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv"
Sep 17 16:58:41.651121 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:41.651073    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ed2282ce-e0bf-48ce-af2a-d9c497d56398-kserve-provision-location\") pod \"isvc-secondary-42374c-predictor-86544c997d-ch6dv\" (UID: \"ed2282ce-e0bf-48ce-af2a-d9c497d56398\") " pod="kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv"
Sep 17 16:58:41.651121 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:41.651111    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ed2282ce-e0bf-48ce-af2a-d9c497d56398-kube-rbac-proxy-sar-config\") pod \"isvc-secondary-42374c-predictor-86544c997d-ch6dv\" (UID: \"ed2282ce-e0bf-48ce-af2a-d9c497d56398\") " pod="kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv"
Sep 17 16:58:41.651227 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:41.651143    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ed2282ce-e0bf-48ce-af2a-d9c497d56398-cabundle-cert\") pod \"isvc-secondary-42374c-predictor-86544c997d-ch6dv\" (UID: \"ed2282ce-e0bf-48ce-af2a-d9c497d56398\") " pod="kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv"
Sep 17 16:58:41.752368 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:41.752329    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ed2282ce-e0bf-48ce-af2a-d9c497d56398-proxy-tls\") pod \"isvc-secondary-42374c-predictor-86544c997d-ch6dv\" (UID: \"ed2282ce-e0bf-48ce-af2a-d9c497d56398\") " pod="kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv"
Sep 17 16:58:41.752368 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:41.752374    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ed2282ce-e0bf-48ce-af2a-d9c497d56398-kserve-provision-location\") pod \"isvc-secondary-42374c-predictor-86544c997d-ch6dv\" (UID: \"ed2282ce-e0bf-48ce-af2a-d9c497d56398\") " pod="kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv"
Sep 17 16:58:41.752635 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:41.752401    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ed2282ce-e0bf-48ce-af2a-d9c497d56398-kube-rbac-proxy-sar-config\") pod \"isvc-secondary-42374c-predictor-86544c997d-ch6dv\" (UID: \"ed2282ce-e0bf-48ce-af2a-d9c497d56398\") " pod="kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv"
Sep 17 16:58:41.752635 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:41.752429    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ed2282ce-e0bf-48ce-af2a-d9c497d56398-cabundle-cert\") pod \"isvc-secondary-42374c-predictor-86544c997d-ch6dv\" (UID: \"ed2282ce-e0bf-48ce-af2a-d9c497d56398\") " pod="kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv"
Sep 17 16:58:41.752635 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:41.752480    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gm8t5\" (UniqueName: \"kubernetes.io/projected/ed2282ce-e0bf-48ce-af2a-d9c497d56398-kube-api-access-gm8t5\") pod \"isvc-secondary-42374c-predictor-86544c997d-ch6dv\" (UID: \"ed2282ce-e0bf-48ce-af2a-d9c497d56398\") " pod="kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv"
Sep 17 16:58:41.752635 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:58:41.752494    2586 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-secondary-42374c-predictor-serving-cert: secret "isvc-secondary-42374c-predictor-serving-cert" not found
Sep 17 16:58:41.752635 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:58:41.752575    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed2282ce-e0bf-48ce-af2a-d9c497d56398-proxy-tls podName:ed2282ce-e0bf-48ce-af2a-d9c497d56398 nodeName:}" failed. No retries permitted until 2026-09-17 16:58:42.252553493 +0000 UTC m=+992.953632133 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/ed2282ce-e0bf-48ce-af2a-d9c497d56398-proxy-tls") pod "isvc-secondary-42374c-predictor-86544c997d-ch6dv" (UID: "ed2282ce-e0bf-48ce-af2a-d9c497d56398") : secret "isvc-secondary-42374c-predictor-serving-cert" not found
Sep 17 16:58:41.752850 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:41.752818    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ed2282ce-e0bf-48ce-af2a-d9c497d56398-kserve-provision-location\") pod \"isvc-secondary-42374c-predictor-86544c997d-ch6dv\" (UID: \"ed2282ce-e0bf-48ce-af2a-d9c497d56398\") " pod="kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv"
Sep 17 16:58:41.753257 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:41.753234    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ed2282ce-e0bf-48ce-af2a-d9c497d56398-cabundle-cert\") pod \"isvc-secondary-42374c-predictor-86544c997d-ch6dv\" (UID: \"ed2282ce-e0bf-48ce-af2a-d9c497d56398\") " pod="kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv"
Sep 17 16:58:41.753359 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:41.753344    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ed2282ce-e0bf-48ce-af2a-d9c497d56398-kube-rbac-proxy-sar-config\") pod \"isvc-secondary-42374c-predictor-86544c997d-ch6dv\" (UID: \"ed2282ce-e0bf-48ce-af2a-d9c497d56398\") " pod="kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv"
Sep 17 16:58:41.761532 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:41.761497    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm8t5\" (UniqueName: \"kubernetes.io/projected/ed2282ce-e0bf-48ce-af2a-d9c497d56398-kube-api-access-gm8t5\") pod \"isvc-secondary-42374c-predictor-86544c997d-ch6dv\" (UID: \"ed2282ce-e0bf-48ce-af2a-d9c497d56398\") " pod="kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv"
Sep 17 16:58:42.257444 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:42.257408    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ed2282ce-e0bf-48ce-af2a-d9c497d56398-proxy-tls\") pod \"isvc-secondary-42374c-predictor-86544c997d-ch6dv\" (UID: \"ed2282ce-e0bf-48ce-af2a-d9c497d56398\") " pod="kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv"
Sep 17 16:58:42.259953 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:42.259933    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ed2282ce-e0bf-48ce-af2a-d9c497d56398-proxy-tls\") pod \"isvc-secondary-42374c-predictor-86544c997d-ch6dv\" (UID: \"ed2282ce-e0bf-48ce-af2a-d9c497d56398\") " pod="kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv"
Sep 17 16:58:42.536739 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:42.536693    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv"
Sep 17 16:58:42.807945 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:42.807919    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv"]
Sep 17 16:58:42.809904 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:58:42.809860    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded2282ce_e0bf_48ce_af2a_d9c497d56398.slice/crio-c926918961c1932ea638f47bcefaaa1186c07158cf0803240f8b31673a5e5ae6 WatchSource:0}: Error finding container c926918961c1932ea638f47bcefaaa1186c07158cf0803240f8b31673a5e5ae6: Status 404 returned error can't find the container with id c926918961c1932ea638f47bcefaaa1186c07158cf0803240f8b31673a5e5ae6
Sep 17 16:58:43.613936 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:43.613901    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv" event={"ID":"ed2282ce-e0bf-48ce-af2a-d9c497d56398","Type":"ContainerStarted","Data":"80913df73ce8743744e9ded86324a3d7b69a55f679d733b09684a916928d8047"}
Sep 17 16:58:43.614104 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:43.613985    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv" event={"ID":"ed2282ce-e0bf-48ce-af2a-d9c497d56398","Type":"ContainerStarted","Data":"c926918961c1932ea638f47bcefaaa1186c07158cf0803240f8b31673a5e5ae6"}
Sep 17 16:58:48.635784 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:48.635756    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-secondary-42374c-predictor-86544c997d-ch6dv_ed2282ce-e0bf-48ce-af2a-d9c497d56398/storage-initializer/0.log"
Sep 17 16:58:48.636173 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:48.635794    2586 generic.go:356] "Generic (PLEG): container finished" podID="ed2282ce-e0bf-48ce-af2a-d9c497d56398" containerID="80913df73ce8743744e9ded86324a3d7b69a55f679d733b09684a916928d8047" exitCode=1
Sep 17 16:58:48.636173 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:48.635889    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv" event={"ID":"ed2282ce-e0bf-48ce-af2a-d9c497d56398","Type":"ContainerDied","Data":"80913df73ce8743744e9ded86324a3d7b69a55f679d733b09684a916928d8047"}
Sep 17 16:58:49.642294 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:49.642269    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-secondary-42374c-predictor-86544c997d-ch6dv_ed2282ce-e0bf-48ce-af2a-d9c497d56398/storage-initializer/0.log"
Sep 17 16:58:49.642684 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:49.642336    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv" event={"ID":"ed2282ce-e0bf-48ce-af2a-d9c497d56398","Type":"ContainerStarted","Data":"7cd0430e8f3b469d8f778f0c9aaab553aa928a0fe6e83c08e2f519b46a62611a"}
Sep 17 16:58:54.667136 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:54.667050    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-secondary-42374c-predictor-86544c997d-ch6dv_ed2282ce-e0bf-48ce-af2a-d9c497d56398/storage-initializer/1.log"
Sep 17 16:58:54.667756 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:54.667738    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-secondary-42374c-predictor-86544c997d-ch6dv_ed2282ce-e0bf-48ce-af2a-d9c497d56398/storage-initializer/0.log"
Sep 17 16:58:54.667810 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:54.667780    2586 generic.go:356] "Generic (PLEG): container finished" podID="ed2282ce-e0bf-48ce-af2a-d9c497d56398" containerID="7cd0430e8f3b469d8f778f0c9aaab553aa928a0fe6e83c08e2f519b46a62611a" exitCode=1
Sep 17 16:58:54.667914 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:54.667863    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv" event={"ID":"ed2282ce-e0bf-48ce-af2a-d9c497d56398","Type":"ContainerDied","Data":"7cd0430e8f3b469d8f778f0c9aaab553aa928a0fe6e83c08e2f519b46a62611a"}
Sep 17 16:58:54.667992 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:54.667980    2586 scope.go:117] "RemoveContainer" containerID="80913df73ce8743744e9ded86324a3d7b69a55f679d733b09684a916928d8047"
Sep 17 16:58:54.668435 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:54.668415    2586 scope.go:117] "RemoveContainer" containerID="80913df73ce8743744e9ded86324a3d7b69a55f679d733b09684a916928d8047"
Sep 17 16:58:54.678118 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:58:54.678097    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"storage-initializer\" with CrashLoopBackOff: \"back-off 10s restarting failed container=storage-initializer pod=isvc-secondary-42374c-predictor-86544c997d-ch6dv_kserve-ci-e2e-test(ed2282ce-e0bf-48ce-af2a-d9c497d56398)\"" pod="kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv" podUID="ed2282ce-e0bf-48ce-af2a-d9c497d56398"
Sep 17 16:58:54.678320 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:58:54.678289    2586 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_storage-initializer_isvc-secondary-42374c-predictor-86544c997d-ch6dv_kserve-ci-e2e-test_ed2282ce-e0bf-48ce-af2a-d9c497d56398_0 in pod sandbox c926918961c1932ea638f47bcefaaa1186c07158cf0803240f8b31673a5e5ae6 from index: no such id: '80913df73ce8743744e9ded86324a3d7b69a55f679d733b09684a916928d8047'" containerID="80913df73ce8743744e9ded86324a3d7b69a55f679d733b09684a916928d8047"
Sep 17 16:58:54.678376 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:54.678330    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80913df73ce8743744e9ded86324a3d7b69a55f679d733b09684a916928d8047"} err="rpc error: code = Unknown desc = failed to delete container k8s_storage-initializer_isvc-secondary-42374c-predictor-86544c997d-ch6dv_kserve-ci-e2e-test_ed2282ce-e0bf-48ce-af2a-d9c497d56398_0 in pod sandbox c926918961c1932ea638f47bcefaaa1186c07158cf0803240f8b31673a5e5ae6 from index: no such id: '80913df73ce8743744e9ded86324a3d7b69a55f679d733b09684a916928d8047'"
Sep 17 16:58:55.673517 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:55.673492    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-secondary-42374c-predictor-86544c997d-ch6dv_ed2282ce-e0bf-48ce-af2a-d9c497d56398/storage-initializer/1.log"
Sep 17 16:58:59.660252 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:59.660218    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv"]
Sep 17 16:58:59.785584 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:59.785558    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-secondary-42374c-predictor-86544c997d-ch6dv_ed2282ce-e0bf-48ce-af2a-d9c497d56398/storage-initializer/1.log"
Sep 17 16:58:59.787957 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:59.787938    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv"
Sep 17 16:58:59.901393 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:59.901363    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ed2282ce-e0bf-48ce-af2a-d9c497d56398-proxy-tls\") pod \"ed2282ce-e0bf-48ce-af2a-d9c497d56398\" (UID: \"ed2282ce-e0bf-48ce-af2a-d9c497d56398\") "
Sep 17 16:58:59.901565 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:59.901428    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gm8t5\" (UniqueName: \"kubernetes.io/projected/ed2282ce-e0bf-48ce-af2a-d9c497d56398-kube-api-access-gm8t5\") pod \"ed2282ce-e0bf-48ce-af2a-d9c497d56398\" (UID: \"ed2282ce-e0bf-48ce-af2a-d9c497d56398\") "
Sep 17 16:58:59.901565 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:59.901486    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ed2282ce-e0bf-48ce-af2a-d9c497d56398-cabundle-cert\") pod \"ed2282ce-e0bf-48ce-af2a-d9c497d56398\" (UID: \"ed2282ce-e0bf-48ce-af2a-d9c497d56398\") "
Sep 17 16:58:59.901565 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:59.901506    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ed2282ce-e0bf-48ce-af2a-d9c497d56398-kserve-provision-location\") pod \"ed2282ce-e0bf-48ce-af2a-d9c497d56398\" (UID: \"ed2282ce-e0bf-48ce-af2a-d9c497d56398\") "
Sep 17 16:58:59.901565 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:59.901525    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ed2282ce-e0bf-48ce-af2a-d9c497d56398-kube-rbac-proxy-sar-config\") pod \"ed2282ce-e0bf-48ce-af2a-d9c497d56398\" (UID: \"ed2282ce-e0bf-48ce-af2a-d9c497d56398\") "
Sep 17 16:58:59.901835 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:59.901803    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed2282ce-e0bf-48ce-af2a-d9c497d56398-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "ed2282ce-e0bf-48ce-af2a-d9c497d56398" (UID: "ed2282ce-e0bf-48ce-af2a-d9c497d56398"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 16:58:59.901945 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:59.901906    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed2282ce-e0bf-48ce-af2a-d9c497d56398-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "ed2282ce-e0bf-48ce-af2a-d9c497d56398" (UID: "ed2282ce-e0bf-48ce-af2a-d9c497d56398"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:58:59.901945 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:59.901936    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed2282ce-e0bf-48ce-af2a-d9c497d56398-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "ed2282ce-e0bf-48ce-af2a-d9c497d56398" (UID: "ed2282ce-e0bf-48ce-af2a-d9c497d56398"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:58:59.903603 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:59.903583    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed2282ce-e0bf-48ce-af2a-d9c497d56398-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ed2282ce-e0bf-48ce-af2a-d9c497d56398" (UID: "ed2282ce-e0bf-48ce-af2a-d9c497d56398"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:58:59.903681 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:58:59.903660    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed2282ce-e0bf-48ce-af2a-d9c497d56398-kube-api-access-gm8t5" (OuterVolumeSpecName: "kube-api-access-gm8t5") pod "ed2282ce-e0bf-48ce-af2a-d9c497d56398" (UID: "ed2282ce-e0bf-48ce-af2a-d9c497d56398"). InnerVolumeSpecName "kube-api-access-gm8t5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:59:00.002250 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:00.002172    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gm8t5\" (UniqueName: \"kubernetes.io/projected/ed2282ce-e0bf-48ce-af2a-d9c497d56398-kube-api-access-gm8t5\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:59:00.002250 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:00.002202    2586 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ed2282ce-e0bf-48ce-af2a-d9c497d56398-cabundle-cert\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:59:00.002250 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:00.002211    2586 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ed2282ce-e0bf-48ce-af2a-d9c497d56398-kserve-provision-location\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:59:00.002250 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:00.002219    2586 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ed2282ce-e0bf-48ce-af2a-d9c497d56398-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:59:00.002250 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:00.002229    2586 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ed2282ce-e0bf-48ce-af2a-d9c497d56398-proxy-tls\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:59:00.699624 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:00.699596    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-secondary-42374c-predictor-86544c997d-ch6dv_ed2282ce-e0bf-48ce-af2a-d9c497d56398/storage-initializer/1.log"
Sep 17 16:59:00.700055 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:00.699681    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv" event={"ID":"ed2282ce-e0bf-48ce-af2a-d9c497d56398","Type":"ContainerDied","Data":"c926918961c1932ea638f47bcefaaa1186c07158cf0803240f8b31673a5e5ae6"}
Sep 17 16:59:00.700055 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:00.699693    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv"
Sep 17 16:59:00.700055 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:00.699738    2586 scope.go:117] "RemoveContainer" containerID="7cd0430e8f3b469d8f778f0c9aaab553aa928a0fe6e83c08e2f519b46a62611a"
Sep 17 16:59:00.734765 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:00.734741    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv"]
Sep 17 16:59:00.740426 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:00.740405    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-secondary-42374c-predictor-86544c997d-ch6dv"]
Sep 17 16:59:01.324323 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:01.324287    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed2282ce-e0bf-48ce-af2a-d9c497d56398" path="/var/lib/kubelet/pods/ed2282ce-e0bf-48ce-af2a-d9c497d56398/volumes"
Sep 17 16:59:04.888671 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:04.888634    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk"]
Sep 17 16:59:04.889075 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:04.888969    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed2282ce-e0bf-48ce-af2a-d9c497d56398" containerName="storage-initializer"
Sep 17 16:59:04.889075 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:04.888983    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed2282ce-e0bf-48ce-af2a-d9c497d56398" containerName="storage-initializer"
Sep 17 16:59:04.889075 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:04.889034    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed2282ce-e0bf-48ce-af2a-d9c497d56398" containerName="storage-initializer"
Sep 17 16:59:04.889184 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:04.889088    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed2282ce-e0bf-48ce-af2a-d9c497d56398" containerName="storage-initializer"
Sep 17 16:59:04.889184 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:04.889094    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed2282ce-e0bf-48ce-af2a-d9c497d56398" containerName="storage-initializer"
Sep 17 16:59:04.889184 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:04.889160    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed2282ce-e0bf-48ce-af2a-d9c497d56398" containerName="storage-initializer"
Sep 17 16:59:04.893422 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:04.893406    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk"
Sep 17 16:59:04.896035 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:04.896011    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Sep 17 16:59:04.896163 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:04.896011    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-init-fail-d41ccc-kube-rbac-proxy-sar-config\""
Sep 17 16:59:04.896163 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:04.896109    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 17 16:59:04.897103 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:04.897086    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"fail-s3-sa-d41ccc-dockercfg-4tpn5\""
Sep 17 16:59:04.897197 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:04.897089    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"fail-s3-secret-d41ccc\""
Sep 17 16:59:04.897197 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:04.897116    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-init-fail-d41ccc-predictor-serving-cert\""
Sep 17 16:59:04.897197 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:04.897152    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 17 16:59:04.903396 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:04.903370    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk"]
Sep 17 16:59:04.945957 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:04.945910    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/71d9dd9e-e496-40b5-9de2-fc380aa44e2c-proxy-tls\") pod \"isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk\" (UID: \"71d9dd9e-e496-40b5-9de2-fc380aa44e2c\") " pod="kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk"
Sep 17 16:59:04.945957 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:04.945957    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/71d9dd9e-e496-40b5-9de2-fc380aa44e2c-kserve-provision-location\") pod \"isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk\" (UID: \"71d9dd9e-e496-40b5-9de2-fc380aa44e2c\") " pod="kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk"
Sep 17 16:59:04.946139 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:04.945984    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/71d9dd9e-e496-40b5-9de2-fc380aa44e2c-kube-rbac-proxy-sar-config\") pod \"isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk\" (UID: \"71d9dd9e-e496-40b5-9de2-fc380aa44e2c\") " pod="kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk"
Sep 17 16:59:04.946139 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:04.946017    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89h6k\" (UniqueName: \"kubernetes.io/projected/71d9dd9e-e496-40b5-9de2-fc380aa44e2c-kube-api-access-89h6k\") pod \"isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk\" (UID: \"71d9dd9e-e496-40b5-9de2-fc380aa44e2c\") " pod="kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk"
Sep 17 16:59:04.946139 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:04.946051    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/71d9dd9e-e496-40b5-9de2-fc380aa44e2c-cabundle-cert\") pod \"isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk\" (UID: \"71d9dd9e-e496-40b5-9de2-fc380aa44e2c\") " pod="kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk"
Sep 17 16:59:05.047230 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:05.047193    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89h6k\" (UniqueName: \"kubernetes.io/projected/71d9dd9e-e496-40b5-9de2-fc380aa44e2c-kube-api-access-89h6k\") pod \"isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk\" (UID: \"71d9dd9e-e496-40b5-9de2-fc380aa44e2c\") " pod="kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk"
Sep 17 16:59:05.047410 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:05.047251    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/71d9dd9e-e496-40b5-9de2-fc380aa44e2c-cabundle-cert\") pod \"isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk\" (UID: \"71d9dd9e-e496-40b5-9de2-fc380aa44e2c\") " pod="kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk"
Sep 17 16:59:05.047410 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:05.047296    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/71d9dd9e-e496-40b5-9de2-fc380aa44e2c-proxy-tls\") pod \"isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk\" (UID: \"71d9dd9e-e496-40b5-9de2-fc380aa44e2c\") " pod="kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk"
Sep 17 16:59:05.047410 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:05.047323    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/71d9dd9e-e496-40b5-9de2-fc380aa44e2c-kserve-provision-location\") pod \"isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk\" (UID: \"71d9dd9e-e496-40b5-9de2-fc380aa44e2c\") " pod="kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk"
Sep 17 16:59:05.047410 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:05.047345    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/71d9dd9e-e496-40b5-9de2-fc380aa44e2c-kube-rbac-proxy-sar-config\") pod \"isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk\" (UID: \"71d9dd9e-e496-40b5-9de2-fc380aa44e2c\") " pod="kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk"
Sep 17 16:59:05.047819 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:05.047795    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/71d9dd9e-e496-40b5-9de2-fc380aa44e2c-kserve-provision-location\") pod \"isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk\" (UID: \"71d9dd9e-e496-40b5-9de2-fc380aa44e2c\") " pod="kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk"
Sep 17 16:59:05.048061 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:05.048043    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/71d9dd9e-e496-40b5-9de2-fc380aa44e2c-cabundle-cert\") pod \"isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk\" (UID: \"71d9dd9e-e496-40b5-9de2-fc380aa44e2c\") " pod="kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk"
Sep 17 16:59:05.048127 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:05.048072    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/71d9dd9e-e496-40b5-9de2-fc380aa44e2c-kube-rbac-proxy-sar-config\") pod \"isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk\" (UID: \"71d9dd9e-e496-40b5-9de2-fc380aa44e2c\") " pod="kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk"
Sep 17 16:59:05.049903 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:05.049867    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/71d9dd9e-e496-40b5-9de2-fc380aa44e2c-proxy-tls\") pod \"isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk\" (UID: \"71d9dd9e-e496-40b5-9de2-fc380aa44e2c\") " pod="kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk"
Sep 17 16:59:05.055283 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:05.055257    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89h6k\" (UniqueName: \"kubernetes.io/projected/71d9dd9e-e496-40b5-9de2-fc380aa44e2c-kube-api-access-89h6k\") pod \"isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk\" (UID: \"71d9dd9e-e496-40b5-9de2-fc380aa44e2c\") " pod="kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk"
Sep 17 16:59:05.205702 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:05.205613    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk"
Sep 17 16:59:05.480063 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:05.479919    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk"]
Sep 17 16:59:05.482762 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:59:05.482735    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71d9dd9e_e496_40b5_9de2_fc380aa44e2c.slice/crio-a604a13314a8de6ef532c5278e641c62ebba42aa9ebd207522623b8ddc2d7e6c WatchSource:0}: Error finding container a604a13314a8de6ef532c5278e641c62ebba42aa9ebd207522623b8ddc2d7e6c: Status 404 returned error can't find the container with id a604a13314a8de6ef532c5278e641c62ebba42aa9ebd207522623b8ddc2d7e6c
Sep 17 16:59:05.723500 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:05.723462    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk" event={"ID":"71d9dd9e-e496-40b5-9de2-fc380aa44e2c","Type":"ContainerStarted","Data":"6f90cd2e3c07325a3893fc572ad9400fafcf208a8afe49dbb9a7aeed36c8c19b"}
Sep 17 16:59:05.723674 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:05.723521    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk" event={"ID":"71d9dd9e-e496-40b5-9de2-fc380aa44e2c","Type":"ContainerStarted","Data":"a604a13314a8de6ef532c5278e641c62ebba42aa9ebd207522623b8ddc2d7e6c"}
Sep 17 16:59:10.745709 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:10.745682    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk_71d9dd9e-e496-40b5-9de2-fc380aa44e2c/storage-initializer/0.log"
Sep 17 16:59:10.746083 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:10.745723    2586 generic.go:356] "Generic (PLEG): container finished" podID="71d9dd9e-e496-40b5-9de2-fc380aa44e2c" containerID="6f90cd2e3c07325a3893fc572ad9400fafcf208a8afe49dbb9a7aeed36c8c19b" exitCode=1
Sep 17 16:59:10.746083 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:10.745805    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk" event={"ID":"71d9dd9e-e496-40b5-9de2-fc380aa44e2c","Type":"ContainerDied","Data":"6f90cd2e3c07325a3893fc572ad9400fafcf208a8afe49dbb9a7aeed36c8c19b"}
Sep 17 16:59:11.752297 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:11.752268    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk_71d9dd9e-e496-40b5-9de2-fc380aa44e2c/storage-initializer/0.log"
Sep 17 16:59:11.752767 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:11.752337    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk" event={"ID":"71d9dd9e-e496-40b5-9de2-fc380aa44e2c","Type":"ContainerStarted","Data":"8077a6799e91a1f892f8b0b33240173c83ea42d200316e96813b1957b6d2e267"}
Sep 17 16:59:13.761318 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:13.761287    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk_71d9dd9e-e496-40b5-9de2-fc380aa44e2c/storage-initializer/1.log"
Sep 17 16:59:13.761786 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:13.761738    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk_71d9dd9e-e496-40b5-9de2-fc380aa44e2c/storage-initializer/0.log"
Sep 17 16:59:13.761786 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:13.761770    2586 generic.go:356] "Generic (PLEG): container finished" podID="71d9dd9e-e496-40b5-9de2-fc380aa44e2c" containerID="8077a6799e91a1f892f8b0b33240173c83ea42d200316e96813b1957b6d2e267" exitCode=1
Sep 17 16:59:13.761924 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:13.761854    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk" event={"ID":"71d9dd9e-e496-40b5-9de2-fc380aa44e2c","Type":"ContainerDied","Data":"8077a6799e91a1f892f8b0b33240173c83ea42d200316e96813b1957b6d2e267"}
Sep 17 16:59:13.762006 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:13.761991    2586 scope.go:117] "RemoveContainer" containerID="6f90cd2e3c07325a3893fc572ad9400fafcf208a8afe49dbb9a7aeed36c8c19b"
Sep 17 16:59:13.762394 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:13.762370    2586 scope.go:117] "RemoveContainer" containerID="6f90cd2e3c07325a3893fc572ad9400fafcf208a8afe49dbb9a7aeed36c8c19b"
Sep 17 16:59:13.772786 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:59:13.772765    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"storage-initializer\" with CrashLoopBackOff: \"back-off 10s restarting failed container=storage-initializer pod=isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk_kserve-ci-e2e-test(71d9dd9e-e496-40b5-9de2-fc380aa44e2c)\"" pod="kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk" podUID="71d9dd9e-e496-40b5-9de2-fc380aa44e2c"
Sep 17 16:59:13.773053 ip-10-0-130-106 kubenswrapper[2586]: E0917 16:59:13.773029    2586 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_storage-initializer_isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk_kserve-ci-e2e-test_71d9dd9e-e496-40b5-9de2-fc380aa44e2c_0 in pod sandbox a604a13314a8de6ef532c5278e641c62ebba42aa9ebd207522623b8ddc2d7e6c from index: no such id: '6f90cd2e3c07325a3893fc572ad9400fafcf208a8afe49dbb9a7aeed36c8c19b'" containerID="6f90cd2e3c07325a3893fc572ad9400fafcf208a8afe49dbb9a7aeed36c8c19b"
Sep 17 16:59:13.773126 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:13.773060    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f90cd2e3c07325a3893fc572ad9400fafcf208a8afe49dbb9a7aeed36c8c19b"} err="rpc error: code = Unknown desc = failed to delete container k8s_storage-initializer_isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk_kserve-ci-e2e-test_71d9dd9e-e496-40b5-9de2-fc380aa44e2c_0 in pod sandbox a604a13314a8de6ef532c5278e641c62ebba42aa9ebd207522623b8ddc2d7e6c from index: no such id: '6f90cd2e3c07325a3893fc572ad9400fafcf208a8afe49dbb9a7aeed36c8c19b'"
Sep 17 16:59:14.767809 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:14.767782    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk_71d9dd9e-e496-40b5-9de2-fc380aa44e2c/storage-initializer/1.log"
Sep 17 16:59:14.869393 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:14.869362    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk"]
Sep 17 16:59:14.994909 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:14.994889    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk_71d9dd9e-e496-40b5-9de2-fc380aa44e2c/storage-initializer/1.log"
Sep 17 16:59:14.998414 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:14.998399    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk"
Sep 17 16:59:15.131684 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:15.131639    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/71d9dd9e-e496-40b5-9de2-fc380aa44e2c-cabundle-cert\") pod \"71d9dd9e-e496-40b5-9de2-fc380aa44e2c\" (UID: \"71d9dd9e-e496-40b5-9de2-fc380aa44e2c\") "
Sep 17 16:59:15.131684 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:15.131694    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/71d9dd9e-e496-40b5-9de2-fc380aa44e2c-proxy-tls\") pod \"71d9dd9e-e496-40b5-9de2-fc380aa44e2c\" (UID: \"71d9dd9e-e496-40b5-9de2-fc380aa44e2c\") "
Sep 17 16:59:15.131985 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:15.131712    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/71d9dd9e-e496-40b5-9de2-fc380aa44e2c-kube-rbac-proxy-sar-config\") pod \"71d9dd9e-e496-40b5-9de2-fc380aa44e2c\" (UID: \"71d9dd9e-e496-40b5-9de2-fc380aa44e2c\") "
Sep 17 16:59:15.131985 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:15.131770    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/71d9dd9e-e496-40b5-9de2-fc380aa44e2c-kserve-provision-location\") pod \"71d9dd9e-e496-40b5-9de2-fc380aa44e2c\" (UID: \"71d9dd9e-e496-40b5-9de2-fc380aa44e2c\") "
Sep 17 16:59:15.131985 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:15.131837    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-89h6k\" (UniqueName: \"kubernetes.io/projected/71d9dd9e-e496-40b5-9de2-fc380aa44e2c-kube-api-access-89h6k\") pod \"71d9dd9e-e496-40b5-9de2-fc380aa44e2c\" (UID: \"71d9dd9e-e496-40b5-9de2-fc380aa44e2c\") "
Sep 17 16:59:15.132133 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:15.132062    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71d9dd9e-e496-40b5-9de2-fc380aa44e2c-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "71d9dd9e-e496-40b5-9de2-fc380aa44e2c" (UID: "71d9dd9e-e496-40b5-9de2-fc380aa44e2c"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 16:59:15.132173 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:15.132136    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71d9dd9e-e496-40b5-9de2-fc380aa44e2c-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "71d9dd9e-e496-40b5-9de2-fc380aa44e2c" (UID: "71d9dd9e-e496-40b5-9de2-fc380aa44e2c"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:59:15.132173 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:15.132141    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71d9dd9e-e496-40b5-9de2-fc380aa44e2c-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "71d9dd9e-e496-40b5-9de2-fc380aa44e2c" (UID: "71d9dd9e-e496-40b5-9de2-fc380aa44e2c"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 16:59:15.133998 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:15.133978    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71d9dd9e-e496-40b5-9de2-fc380aa44e2c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "71d9dd9e-e496-40b5-9de2-fc380aa44e2c" (UID: "71d9dd9e-e496-40b5-9de2-fc380aa44e2c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 16:59:15.134159 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:15.134139    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71d9dd9e-e496-40b5-9de2-fc380aa44e2c-kube-api-access-89h6k" (OuterVolumeSpecName: "kube-api-access-89h6k") pod "71d9dd9e-e496-40b5-9de2-fc380aa44e2c" (UID: "71d9dd9e-e496-40b5-9de2-fc380aa44e2c"). InnerVolumeSpecName "kube-api-access-89h6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 16:59:15.232997 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:15.232967    2586 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/71d9dd9e-e496-40b5-9de2-fc380aa44e2c-cabundle-cert\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:59:15.232997 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:15.232993    2586 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/71d9dd9e-e496-40b5-9de2-fc380aa44e2c-proxy-tls\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:59:15.233146 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:15.233004    2586 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/71d9dd9e-e496-40b5-9de2-fc380aa44e2c-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:59:15.233146 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:15.233015    2586 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/71d9dd9e-e496-40b5-9de2-fc380aa44e2c-kserve-provision-location\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:59:15.233146 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:15.233024    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89h6k\" (UniqueName: \"kubernetes.io/projected/71d9dd9e-e496-40b5-9de2-fc380aa44e2c-kube-api-access-89h6k\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 16:59:15.773456 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:15.773429    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk_71d9dd9e-e496-40b5-9de2-fc380aa44e2c/storage-initializer/1.log"
Sep 17 16:59:15.773841 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:15.773509    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk" event={"ID":"71d9dd9e-e496-40b5-9de2-fc380aa44e2c","Type":"ContainerDied","Data":"a604a13314a8de6ef532c5278e641c62ebba42aa9ebd207522623b8ddc2d7e6c"}
Sep 17 16:59:15.773841 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:15.773520    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk"
Sep 17 16:59:15.773841 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:15.773566    2586 scope.go:117] "RemoveContainer" containerID="8077a6799e91a1f892f8b0b33240173c83ea42d200316e96813b1957b6d2e267"
Sep 17 16:59:15.805413 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:15.805386    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk"]
Sep 17 16:59:15.807897 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:15.807856    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-init-fail-d41ccc-predictor-7cc64b44bd-2k4qk"]
Sep 17 16:59:17.323868 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:17.323835    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71d9dd9e-e496-40b5-9de2-fc380aa44e2c" path="/var/lib/kubelet/pods/71d9dd9e-e496-40b5-9de2-fc380aa44e2c/volumes"
Sep 17 16:59:20.090319 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.090283    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w"]
Sep 17 16:59:20.090690 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.090577    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71d9dd9e-e496-40b5-9de2-fc380aa44e2c" containerName="storage-initializer"
Sep 17 16:59:20.090690 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.090588    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="71d9dd9e-e496-40b5-9de2-fc380aa44e2c" containerName="storage-initializer"
Sep 17 16:59:20.090690 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.090648    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="71d9dd9e-e496-40b5-9de2-fc380aa44e2c" containerName="storage-initializer"
Sep 17 16:59:20.090804 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.090705    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71d9dd9e-e496-40b5-9de2-fc380aa44e2c" containerName="storage-initializer"
Sep 17 16:59:20.090804 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.090712    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="71d9dd9e-e496-40b5-9de2-fc380aa44e2c" containerName="storage-initializer"
Sep 17 16:59:20.090804 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.090751    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="71d9dd9e-e496-40b5-9de2-fc380aa44e2c" containerName="storage-initializer"
Sep 17 16:59:20.095023 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.095005    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w"
Sep 17 16:59:20.097451 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.097431    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"raw-sklearn-f0e12-kube-rbac-proxy-sar-config\""
Sep 17 16:59:20.097571 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.097521    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 17 16:59:20.097703 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.097682    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"raw-sklearn-f0e12-predictor-serving-cert\""
Sep 17 16:59:20.097773 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.097751    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Sep 17 16:59:20.097773 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.097763    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 17 16:59:20.098003 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.097988    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-tlgx8\""
Sep 17 16:59:20.098080 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.097988    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Sep 17 16:59:20.102612 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.102586    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w"]
Sep 17 16:59:20.276033 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.276001    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f84fl\" (UniqueName: \"kubernetes.io/projected/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9-kube-api-access-f84fl\") pod \"raw-sklearn-f0e12-predictor-8656cc4776-v9r9w\" (UID: \"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9\") " pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w"
Sep 17 16:59:20.276212 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.276038    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9-kserve-provision-location\") pod \"raw-sklearn-f0e12-predictor-8656cc4776-v9r9w\" (UID: \"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9\") " pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w"
Sep 17 16:59:20.276212 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.276062    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9-proxy-tls\") pod \"raw-sklearn-f0e12-predictor-8656cc4776-v9r9w\" (UID: \"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9\") " pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w"
Sep 17 16:59:20.276212 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.276121    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-f0e12-predictor-8656cc4776-v9r9w\" (UID: \"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9\") " pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w"
Sep 17 16:59:20.276212 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.276196    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9-cabundle-cert\") pod \"raw-sklearn-f0e12-predictor-8656cc4776-v9r9w\" (UID: \"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9\") " pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w"
Sep 17 16:59:20.376766 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.376689    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f84fl\" (UniqueName: \"kubernetes.io/projected/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9-kube-api-access-f84fl\") pod \"raw-sklearn-f0e12-predictor-8656cc4776-v9r9w\" (UID: \"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9\") " pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w"
Sep 17 16:59:20.376766 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.376725    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9-kserve-provision-location\") pod \"raw-sklearn-f0e12-predictor-8656cc4776-v9r9w\" (UID: \"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9\") " pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w"
Sep 17 16:59:20.376766 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.376749    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9-proxy-tls\") pod \"raw-sklearn-f0e12-predictor-8656cc4776-v9r9w\" (UID: \"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9\") " pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w"
Sep 17 16:59:20.376766 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.376770    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-f0e12-predictor-8656cc4776-v9r9w\" (UID: \"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9\") " pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w"
Sep 17 16:59:20.377113 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.376800    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9-cabundle-cert\") pod \"raw-sklearn-f0e12-predictor-8656cc4776-v9r9w\" (UID: \"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9\") " pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w"
Sep 17 16:59:20.377182 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.377167    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9-kserve-provision-location\") pod \"raw-sklearn-f0e12-predictor-8656cc4776-v9r9w\" (UID: \"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9\") " pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w"
Sep 17 16:59:20.377541 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.377516    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9-cabundle-cert\") pod \"raw-sklearn-f0e12-predictor-8656cc4776-v9r9w\" (UID: \"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9\") " pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w"
Sep 17 16:59:20.377625 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.377527    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-f0e12-predictor-8656cc4776-v9r9w\" (UID: \"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9\") " pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w"
Sep 17 16:59:20.379389 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.379371    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9-proxy-tls\") pod \"raw-sklearn-f0e12-predictor-8656cc4776-v9r9w\" (UID: \"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9\") " pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w"
Sep 17 16:59:20.386824 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.386803    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f84fl\" (UniqueName: \"kubernetes.io/projected/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9-kube-api-access-f84fl\") pod \"raw-sklearn-f0e12-predictor-8656cc4776-v9r9w\" (UID: \"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9\") " pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w"
Sep 17 16:59:20.405821 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.405793    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w"
Sep 17 16:59:20.679097 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.679072    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w"]
Sep 17 16:59:20.681640 ip-10-0-130-106 kubenswrapper[2586]: W0917 16:59:20.681614    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a34bc2b_b7f8_4c3d_8ef3_757dbca0a5a9.slice/crio-b59438a3c1c921efcb521b035d4446306e2a91d30a42790f6df33d0c22aa5c85 WatchSource:0}: Error finding container b59438a3c1c921efcb521b035d4446306e2a91d30a42790f6df33d0c22aa5c85: Status 404 returned error can't find the container with id b59438a3c1c921efcb521b035d4446306e2a91d30a42790f6df33d0c22aa5c85
Sep 17 16:59:20.796431 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.796400    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w" event={"ID":"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9","Type":"ContainerStarted","Data":"ffd6f449dbfb9cda8aeb9df98585b14fe33a13da535599cb7bf602a08232eb7b"}
Sep 17 16:59:20.796586 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:20.796471    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w" event={"ID":"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9","Type":"ContainerStarted","Data":"b59438a3c1c921efcb521b035d4446306e2a91d30a42790f6df33d0c22aa5c85"}
Sep 17 16:59:24.814621 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:24.814585    2586 generic.go:356] "Generic (PLEG): container finished" podID="4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9" containerID="ffd6f449dbfb9cda8aeb9df98585b14fe33a13da535599cb7bf602a08232eb7b" exitCode=0
Sep 17 16:59:24.815008 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:24.814660    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w" event={"ID":"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9","Type":"ContainerDied","Data":"ffd6f449dbfb9cda8aeb9df98585b14fe33a13da535599cb7bf602a08232eb7b"}
Sep 17 16:59:25.820795 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:25.820757    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w" event={"ID":"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9","Type":"ContainerStarted","Data":"56f75766b1572fc9f9f057204f770b5794313df7f2b325c88e51e24081fd3f3f"}
Sep 17 16:59:25.821209 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:25.820825    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w" event={"ID":"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9","Type":"ContainerStarted","Data":"a5e3488e7db7f80547a96189910562b97304abc6a6bb150c172f9f0cf70aace0"}
Sep 17 16:59:25.821209 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:25.821041    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w"
Sep 17 16:59:25.841023 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:25.840976    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w" podStartSLOduration=1.99571318 podStartE2EDuration="5.840962433s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.845249253s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:59:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:59:25.838488316 +0000 UTC m=+1036.539566979" watchObservedRunningTime="2026-09-17 16:59:25.840962433 +0000 UTC m=+1036.542041096"
Sep 17 16:59:26.825431 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:26.825399    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w"
Sep 17 16:59:26.826771 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:26.826742    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w" podUID="4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 17 16:59:27.831337 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:27.831294    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w" podUID="4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 17 16:59:32.835771 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:32.835740    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w"
Sep 17 16:59:32.836262 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:32.836238    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w" podUID="4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 17 16:59:42.836550 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:42.836506    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w" podUID="4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 17 16:59:52.836554 ip-10-0-130-106 kubenswrapper[2586]: I0917 16:59:52.836458    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w" podUID="4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 17 17:00:02.836364 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:02.836321    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w" podUID="4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 17 17:00:12.836707 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:12.836662    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w" podUID="4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 17 17:00:22.836699 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:22.836628    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w" podUID="4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 17 17:00:32.836287 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:32.836246    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w" podUID="4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 17 17:00:42.837561 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:42.837528    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w"
Sep 17 17:00:50.161370 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:50.161325    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w"]
Sep 17 17:00:50.161840 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:50.161593    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w" podUID="4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9" containerName="kserve-container" containerID="cri-o://a5e3488e7db7f80547a96189910562b97304abc6a6bb150c172f9f0cf70aace0" gracePeriod=30
Sep 17 17:00:50.161840 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:50.161605    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w" podUID="4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9" containerName="kube-rbac-proxy" containerID="cri-o://56f75766b1572fc9f9f057204f770b5794313df7f2b325c88e51e24081fd3f3f" gracePeriod=30
Sep 17 17:00:51.193326 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:51.193294    2586 generic.go:356] "Generic (PLEG): container finished" podID="4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9" containerID="56f75766b1572fc9f9f057204f770b5794313df7f2b325c88e51e24081fd3f3f" exitCode=2
Sep 17 17:00:51.193702 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:51.193365    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w" event={"ID":"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9","Type":"ContainerDied","Data":"56f75766b1572fc9f9f057204f770b5794313df7f2b325c88e51e24081fd3f3f"}
Sep 17 17:00:52.832416 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:52.832366    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w" podUID="4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.35:8643/healthz\": dial tcp 10.133.0.35:8643: connect: connection refused"
Sep 17 17:00:52.836692 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:52.836664    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w" podUID="4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 17 17:00:54.808372 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:54.808346    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w"
Sep 17 17:00:54.906169 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:54.906071    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9-kserve-provision-location\") pod \"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9\" (UID: \"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9\") "
Sep 17 17:00:54.906169 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:54.906118    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9-proxy-tls\") pod \"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9\" (UID: \"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9\") "
Sep 17 17:00:54.906399 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:54.906176    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f84fl\" (UniqueName: \"kubernetes.io/projected/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9-kube-api-access-f84fl\") pod \"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9\" (UID: \"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9\") "
Sep 17 17:00:54.906399 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:54.906196    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9-cabundle-cert\") pod \"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9\" (UID: \"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9\") "
Sep 17 17:00:54.906399 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:54.906227    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9-kube-rbac-proxy-sar-config\") pod \"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9\" (UID: \"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9\") "
Sep 17 17:00:54.906561 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:54.906426    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9" (UID: "4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:00:54.906617 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:54.906555    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9" (UID: "4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:00:54.906657 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:54.906629    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9" (UID: "4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:00:54.908496 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:54.908473    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9" (UID: "4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:00:54.908620 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:54.908598    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9-kube-api-access-f84fl" (OuterVolumeSpecName: "kube-api-access-f84fl") pod "4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9" (UID: "4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9"). InnerVolumeSpecName "kube-api-access-f84fl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:00:55.007411 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:55.007378    2586 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9-kserve-provision-location\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 17:00:55.007411 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:55.007404    2586 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9-proxy-tls\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 17:00:55.007411 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:55.007415    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f84fl\" (UniqueName: \"kubernetes.io/projected/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9-kube-api-access-f84fl\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 17:00:55.007636 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:55.007424    2586 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9-cabundle-cert\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 17:00:55.007636 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:55.007433    2586 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 17:00:55.212010 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:55.211929    2586 generic.go:356] "Generic (PLEG): container finished" podID="4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9" containerID="a5e3488e7db7f80547a96189910562b97304abc6a6bb150c172f9f0cf70aace0" exitCode=0
Sep 17 17:00:55.212010 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:55.211984    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w"
Sep 17 17:00:55.212201 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:55.212007    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w" event={"ID":"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9","Type":"ContainerDied","Data":"a5e3488e7db7f80547a96189910562b97304abc6a6bb150c172f9f0cf70aace0"}
Sep 17 17:00:55.212201 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:55.212065    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w" event={"ID":"4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9","Type":"ContainerDied","Data":"b59438a3c1c921efcb521b035d4446306e2a91d30a42790f6df33d0c22aa5c85"}
Sep 17 17:00:55.212201 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:55.212091    2586 scope.go:117] "RemoveContainer" containerID="56f75766b1572fc9f9f057204f770b5794313df7f2b325c88e51e24081fd3f3f"
Sep 17 17:00:55.221085 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:55.221050    2586 scope.go:117] "RemoveContainer" containerID="a5e3488e7db7f80547a96189910562b97304abc6a6bb150c172f9f0cf70aace0"
Sep 17 17:00:55.228586 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:55.228571    2586 scope.go:117] "RemoveContainer" containerID="ffd6f449dbfb9cda8aeb9df98585b14fe33a13da535599cb7bf602a08232eb7b"
Sep 17 17:00:55.235415 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:55.235395    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w"]
Sep 17 17:00:55.238458 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:55.238398    2586 scope.go:117] "RemoveContainer" containerID="56f75766b1572fc9f9f057204f770b5794313df7f2b325c88e51e24081fd3f3f"
Sep 17 17:00:55.239203 ip-10-0-130-106 kubenswrapper[2586]: E0917 17:00:55.239144    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56f75766b1572fc9f9f057204f770b5794313df7f2b325c88e51e24081fd3f3f\": container with ID starting with 56f75766b1572fc9f9f057204f770b5794313df7f2b325c88e51e24081fd3f3f not found: ID does not exist" containerID="56f75766b1572fc9f9f057204f770b5794313df7f2b325c88e51e24081fd3f3f"
Sep 17 17:00:55.239203 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:55.239183    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56f75766b1572fc9f9f057204f770b5794313df7f2b325c88e51e24081fd3f3f"} err="failed to get container status \"56f75766b1572fc9f9f057204f770b5794313df7f2b325c88e51e24081fd3f3f\": rpc error: code = NotFound desc = could not find container \"56f75766b1572fc9f9f057204f770b5794313df7f2b325c88e51e24081fd3f3f\": container with ID starting with 56f75766b1572fc9f9f057204f770b5794313df7f2b325c88e51e24081fd3f3f not found: ID does not exist"
Sep 17 17:00:55.239402 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:55.239218    2586 scope.go:117] "RemoveContainer" containerID="a5e3488e7db7f80547a96189910562b97304abc6a6bb150c172f9f0cf70aace0"
Sep 17 17:00:55.239864 ip-10-0-130-106 kubenswrapper[2586]: E0917 17:00:55.239843    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5e3488e7db7f80547a96189910562b97304abc6a6bb150c172f9f0cf70aace0\": container with ID starting with a5e3488e7db7f80547a96189910562b97304abc6a6bb150c172f9f0cf70aace0 not found: ID does not exist" containerID="a5e3488e7db7f80547a96189910562b97304abc6a6bb150c172f9f0cf70aace0"
Sep 17 17:00:55.239864 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:55.239867    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5e3488e7db7f80547a96189910562b97304abc6a6bb150c172f9f0cf70aace0"} err="failed to get container status \"a5e3488e7db7f80547a96189910562b97304abc6a6bb150c172f9f0cf70aace0\": rpc error: code = NotFound desc = could not find container \"a5e3488e7db7f80547a96189910562b97304abc6a6bb150c172f9f0cf70aace0\": container with ID starting with a5e3488e7db7f80547a96189910562b97304abc6a6bb150c172f9f0cf70aace0 not found: ID does not exist"
Sep 17 17:00:55.239864 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:55.239921    2586 scope.go:117] "RemoveContainer" containerID="ffd6f449dbfb9cda8aeb9df98585b14fe33a13da535599cb7bf602a08232eb7b"
Sep 17 17:00:55.240236 ip-10-0-130-106 kubenswrapper[2586]: E0917 17:00:55.240219    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffd6f449dbfb9cda8aeb9df98585b14fe33a13da535599cb7bf602a08232eb7b\": container with ID starting with ffd6f449dbfb9cda8aeb9df98585b14fe33a13da535599cb7bf602a08232eb7b not found: ID does not exist" containerID="ffd6f449dbfb9cda8aeb9df98585b14fe33a13da535599cb7bf602a08232eb7b"
Sep 17 17:00:55.240305 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:55.240243    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffd6f449dbfb9cda8aeb9df98585b14fe33a13da535599cb7bf602a08232eb7b"} err="failed to get container status \"ffd6f449dbfb9cda8aeb9df98585b14fe33a13da535599cb7bf602a08232eb7b\": rpc error: code = NotFound desc = could not find container \"ffd6f449dbfb9cda8aeb9df98585b14fe33a13da535599cb7bf602a08232eb7b\": container with ID starting with ffd6f449dbfb9cda8aeb9df98585b14fe33a13da535599cb7bf602a08232eb7b not found: ID does not exist"
Sep 17 17:00:55.240775 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:55.240756    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-f0e12-predictor-8656cc4776-v9r9w"]
Sep 17 17:00:55.324920 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:00:55.324857    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9" path="/var/lib/kubelet/pods/4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9/volumes"
Sep 17 17:02:07.331150 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.331119    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv"]
Sep 17 17:02:07.331623 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.331442    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9" containerName="storage-initializer"
Sep 17 17:02:07.331623 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.331457    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9" containerName="storage-initializer"
Sep 17 17:02:07.331623 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.331468    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9" containerName="kube-rbac-proxy"
Sep 17 17:02:07.331623 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.331474    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9" containerName="kube-rbac-proxy"
Sep 17 17:02:07.331623 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.331484    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9" containerName="kserve-container"
Sep 17 17:02:07.331623 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.331490    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9" containerName="kserve-container"
Sep 17 17:02:07.331623 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.331542    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9" containerName="kserve-container"
Sep 17 17:02:07.331623 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.331554    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a34bc2b-b7f8-4c3d-8ef3-757dbca0a5a9" containerName="kube-rbac-proxy"
Sep 17 17:02:07.334726 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.334703    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv"
Sep 17 17:02:07.337645 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.337621    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 17 17:02:07.337897 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.337655    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-create-predictor-serving-cert\""
Sep 17 17:02:07.338014 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.337691    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-create-kube-rbac-proxy-sar-config\""
Sep 17 17:02:07.338014 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.337709    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Sep 17 17:02:07.338014 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.337727    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 17 17:02:07.338695 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.338678    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Sep 17 17:02:07.338756 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.338714    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-tlgx8\""
Sep 17 17:02:07.345021 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.345001    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv"]
Sep 17 17:02:07.371241 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.371212    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2cfa55ac-7c33-4c71-a975-0f17c954afc6-proxy-tls\") pod \"isvc-canary-create-predictor-8cb968449-cxzjv\" (UID: \"2cfa55ac-7c33-4c71-a975-0f17c954afc6\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv"
Sep 17 17:02:07.371241 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.371244    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2cfa55ac-7c33-4c71-a975-0f17c954afc6-cabundle-cert\") pod \"isvc-canary-create-predictor-8cb968449-cxzjv\" (UID: \"2cfa55ac-7c33-4c71-a975-0f17c954afc6\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv"
Sep 17 17:02:07.371446 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.371284    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2cfa55ac-7c33-4c71-a975-0f17c954afc6-kserve-provision-location\") pod \"isvc-canary-create-predictor-8cb968449-cxzjv\" (UID: \"2cfa55ac-7c33-4c71-a975-0f17c954afc6\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv"
Sep 17 17:02:07.371446 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.371329    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt9mn\" (UniqueName: \"kubernetes.io/projected/2cfa55ac-7c33-4c71-a975-0f17c954afc6-kube-api-access-pt9mn\") pod \"isvc-canary-create-predictor-8cb968449-cxzjv\" (UID: \"2cfa55ac-7c33-4c71-a975-0f17c954afc6\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv"
Sep 17 17:02:07.371446 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.371353    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2cfa55ac-7c33-4c71-a975-0f17c954afc6-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-8cb968449-cxzjv\" (UID: \"2cfa55ac-7c33-4c71-a975-0f17c954afc6\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv"
Sep 17 17:02:07.472015 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.471977    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2cfa55ac-7c33-4c71-a975-0f17c954afc6-proxy-tls\") pod \"isvc-canary-create-predictor-8cb968449-cxzjv\" (UID: \"2cfa55ac-7c33-4c71-a975-0f17c954afc6\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv"
Sep 17 17:02:07.472015 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.472011    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2cfa55ac-7c33-4c71-a975-0f17c954afc6-cabundle-cert\") pod \"isvc-canary-create-predictor-8cb968449-cxzjv\" (UID: \"2cfa55ac-7c33-4c71-a975-0f17c954afc6\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv"
Sep 17 17:02:07.472257 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.472044    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2cfa55ac-7c33-4c71-a975-0f17c954afc6-kserve-provision-location\") pod \"isvc-canary-create-predictor-8cb968449-cxzjv\" (UID: \"2cfa55ac-7c33-4c71-a975-0f17c954afc6\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv"
Sep 17 17:02:07.472257 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.472061    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt9mn\" (UniqueName: \"kubernetes.io/projected/2cfa55ac-7c33-4c71-a975-0f17c954afc6-kube-api-access-pt9mn\") pod \"isvc-canary-create-predictor-8cb968449-cxzjv\" (UID: \"2cfa55ac-7c33-4c71-a975-0f17c954afc6\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv"
Sep 17 17:02:07.472257 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.472078    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2cfa55ac-7c33-4c71-a975-0f17c954afc6-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-8cb968449-cxzjv\" (UID: \"2cfa55ac-7c33-4c71-a975-0f17c954afc6\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv"
Sep 17 17:02:07.472257 ip-10-0-130-106 kubenswrapper[2586]: E0917 17:02:07.472160    2586 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-canary-create-predictor-serving-cert: secret "isvc-canary-create-predictor-serving-cert" not found
Sep 17 17:02:07.472257 ip-10-0-130-106 kubenswrapper[2586]: E0917 17:02:07.472235    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cfa55ac-7c33-4c71-a975-0f17c954afc6-proxy-tls podName:2cfa55ac-7c33-4c71-a975-0f17c954afc6 nodeName:}" failed. No retries permitted until 2026-09-17 17:02:07.972213636 +0000 UTC m=+1198.673292293 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/2cfa55ac-7c33-4c71-a975-0f17c954afc6-proxy-tls") pod "isvc-canary-create-predictor-8cb968449-cxzjv" (UID: "2cfa55ac-7c33-4c71-a975-0f17c954afc6") : secret "isvc-canary-create-predictor-serving-cert" not found
Sep 17 17:02:07.472489 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.472442    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2cfa55ac-7c33-4c71-a975-0f17c954afc6-kserve-provision-location\") pod \"isvc-canary-create-predictor-8cb968449-cxzjv\" (UID: \"2cfa55ac-7c33-4c71-a975-0f17c954afc6\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv"
Sep 17 17:02:07.472731 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.472714    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2cfa55ac-7c33-4c71-a975-0f17c954afc6-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-8cb968449-cxzjv\" (UID: \"2cfa55ac-7c33-4c71-a975-0f17c954afc6\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv"
Sep 17 17:02:07.472771 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.472727    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2cfa55ac-7c33-4c71-a975-0f17c954afc6-cabundle-cert\") pod \"isvc-canary-create-predictor-8cb968449-cxzjv\" (UID: \"2cfa55ac-7c33-4c71-a975-0f17c954afc6\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv"
Sep 17 17:02:07.480766 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.480740    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt9mn\" (UniqueName: \"kubernetes.io/projected/2cfa55ac-7c33-4c71-a975-0f17c954afc6-kube-api-access-pt9mn\") pod \"isvc-canary-create-predictor-8cb968449-cxzjv\" (UID: \"2cfa55ac-7c33-4c71-a975-0f17c954afc6\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv"
Sep 17 17:02:07.976117 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.976079    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2cfa55ac-7c33-4c71-a975-0f17c954afc6-proxy-tls\") pod \"isvc-canary-create-predictor-8cb968449-cxzjv\" (UID: \"2cfa55ac-7c33-4c71-a975-0f17c954afc6\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv"
Sep 17 17:02:07.978620 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:07.978591    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2cfa55ac-7c33-4c71-a975-0f17c954afc6-proxy-tls\") pod \"isvc-canary-create-predictor-8cb968449-cxzjv\" (UID: \"2cfa55ac-7c33-4c71-a975-0f17c954afc6\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv"
Sep 17 17:02:08.245701 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:08.245601    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv"
Sep 17 17:02:08.520212 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:08.520196    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv"]
Sep 17 17:02:08.522382 ip-10-0-130-106 kubenswrapper[2586]: W0917 17:02:08.522353    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cfa55ac_7c33_4c71_a975_0f17c954afc6.slice/crio-04b395b3bfc11946afc8c56109ba5c7b6f5f10fbd4fe604aa568f4431059d36e WatchSource:0}: Error finding container 04b395b3bfc11946afc8c56109ba5c7b6f5f10fbd4fe604aa568f4431059d36e: Status 404 returned error can't find the container with id 04b395b3bfc11946afc8c56109ba5c7b6f5f10fbd4fe604aa568f4431059d36e
Sep 17 17:02:09.519600 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:09.519564    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv" event={"ID":"2cfa55ac-7c33-4c71-a975-0f17c954afc6","Type":"ContainerStarted","Data":"317a94eeb6715a0e7f25a8e7fc7969c165d72bba1048d74159d0b3f9ecbe7480"}
Sep 17 17:02:09.519784 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:09.519626    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv" event={"ID":"2cfa55ac-7c33-4c71-a975-0f17c954afc6","Type":"ContainerStarted","Data":"04b395b3bfc11946afc8c56109ba5c7b6f5f10fbd4fe604aa568f4431059d36e"}
Sep 17 17:02:12.534188 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:12.534149    2586 generic.go:356] "Generic (PLEG): container finished" podID="2cfa55ac-7c33-4c71-a975-0f17c954afc6" containerID="317a94eeb6715a0e7f25a8e7fc7969c165d72bba1048d74159d0b3f9ecbe7480" exitCode=0
Sep 17 17:02:12.534572 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:12.534188    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv" event={"ID":"2cfa55ac-7c33-4c71-a975-0f17c954afc6","Type":"ContainerDied","Data":"317a94eeb6715a0e7f25a8e7fc7969c165d72bba1048d74159d0b3f9ecbe7480"}
Sep 17 17:02:12.963485 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:12.963374    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22fdn_68559fca-75ed-4ee8-bedd-57b04fbd3f4d/ovn-acl-logging/0.log"
Sep 17 17:02:12.963485 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:12.963426    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22fdn_68559fca-75ed-4ee8-bedd-57b04fbd3f4d/ovn-acl-logging/0.log"
Sep 17 17:02:13.540778 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:13.540746    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv" event={"ID":"2cfa55ac-7c33-4c71-a975-0f17c954afc6","Type":"ContainerStarted","Data":"90c82047e691380acae0f4b4f138aee12a29b114b3f98238b3388f529b63d905"}
Sep 17 17:02:13.541183 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:13.540805    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv" event={"ID":"2cfa55ac-7c33-4c71-a975-0f17c954afc6","Type":"ContainerStarted","Data":"c00aa222d194301eda76426c0d2df92a8d17c32144f684a6f45b3c881dbf39f2"}
Sep 17 17:02:13.541183 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:13.541025    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv"
Sep 17 17:02:13.562608 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:13.562558    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv" podStartSLOduration=2.672661364 podStartE2EDuration="6.562539381s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.889878017s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:02:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:02:13.561191308 +0000 UTC m=+1204.262269981" watchObservedRunningTime="2026-09-17 17:02:13.562539381 +0000 UTC m=+1204.263618046"
Sep 17 17:02:14.545616 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:14.545582    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv"
Sep 17 17:02:14.547217 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:14.547183    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv" podUID="2cfa55ac-7c33-4c71-a975-0f17c954afc6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 17 17:02:15.549740 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:15.549698    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv" podUID="2cfa55ac-7c33-4c71-a975-0f17c954afc6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 17 17:02:20.554207 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:20.554179    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv"
Sep 17 17:02:20.554768 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:20.554740    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv" podUID="2cfa55ac-7c33-4c71-a975-0f17c954afc6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 17 17:02:30.554813 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:30.554772    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv" podUID="2cfa55ac-7c33-4c71-a975-0f17c954afc6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 17 17:02:40.555557 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:40.555519    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv" podUID="2cfa55ac-7c33-4c71-a975-0f17c954afc6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 17 17:02:50.555409 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:02:50.555366    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv" podUID="2cfa55ac-7c33-4c71-a975-0f17c954afc6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 17 17:03:00.555367 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:00.555277    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv" podUID="2cfa55ac-7c33-4c71-a975-0f17c954afc6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 17 17:03:10.555013 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:10.554975    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv" podUID="2cfa55ac-7c33-4c71-a975-0f17c954afc6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 17 17:03:20.554856 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:20.554805    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv" podUID="2cfa55ac-7c33-4c71-a975-0f17c954afc6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 17 17:03:24.320115 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:24.320069    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv" podUID="2cfa55ac-7c33-4c71-a975-0f17c954afc6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 17 17:03:34.321085 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:34.321052    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv"
Sep 17 17:03:37.316193 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:37.316158    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv"]
Sep 17 17:03:37.316604 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:37.316405    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv" podUID="2cfa55ac-7c33-4c71-a975-0f17c954afc6" containerName="kserve-container" containerID="cri-o://c00aa222d194301eda76426c0d2df92a8d17c32144f684a6f45b3c881dbf39f2" gracePeriod=30
Sep 17 17:03:37.316604 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:37.316492    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv" podUID="2cfa55ac-7c33-4c71-a975-0f17c954afc6" containerName="kube-rbac-proxy" containerID="cri-o://90c82047e691380acae0f4b4f138aee12a29b114b3f98238b3388f529b63d905" gracePeriod=30
Sep 17 17:03:37.901867 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:37.901829    2586 generic.go:356] "Generic (PLEG): container finished" podID="2cfa55ac-7c33-4c71-a975-0f17c954afc6" containerID="90c82047e691380acae0f4b4f138aee12a29b114b3f98238b3388f529b63d905" exitCode=2
Sep 17 17:03:37.901867 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:37.901886    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv" event={"ID":"2cfa55ac-7c33-4c71-a975-0f17c954afc6","Type":"ContainerDied","Data":"90c82047e691380acae0f4b4f138aee12a29b114b3f98238b3388f529b63d905"}
Sep 17 17:03:40.550554 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:40.550503    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv" podUID="2cfa55ac-7c33-4c71-a975-0f17c954afc6" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.36:8643/healthz\": dial tcp 10.133.0.36:8643: connect: connection refused"
Sep 17 17:03:42.063040 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.063015    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv"
Sep 17 17:03:42.141582 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.141496    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2cfa55ac-7c33-4c71-a975-0f17c954afc6-kserve-provision-location\") pod \"2cfa55ac-7c33-4c71-a975-0f17c954afc6\" (UID: \"2cfa55ac-7c33-4c71-a975-0f17c954afc6\") "
Sep 17 17:03:42.141582 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.141544    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2cfa55ac-7c33-4c71-a975-0f17c954afc6-kube-rbac-proxy-sar-config\") pod \"2cfa55ac-7c33-4c71-a975-0f17c954afc6\" (UID: \"2cfa55ac-7c33-4c71-a975-0f17c954afc6\") "
Sep 17 17:03:42.141784 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.141605    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2cfa55ac-7c33-4c71-a975-0f17c954afc6-proxy-tls\") pod \"2cfa55ac-7c33-4c71-a975-0f17c954afc6\" (UID: \"2cfa55ac-7c33-4c71-a975-0f17c954afc6\") "
Sep 17 17:03:42.141784 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.141637    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pt9mn\" (UniqueName: \"kubernetes.io/projected/2cfa55ac-7c33-4c71-a975-0f17c954afc6-kube-api-access-pt9mn\") pod \"2cfa55ac-7c33-4c71-a975-0f17c954afc6\" (UID: \"2cfa55ac-7c33-4c71-a975-0f17c954afc6\") "
Sep 17 17:03:42.141784 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.141657    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2cfa55ac-7c33-4c71-a975-0f17c954afc6-cabundle-cert\") pod \"2cfa55ac-7c33-4c71-a975-0f17c954afc6\" (UID: \"2cfa55ac-7c33-4c71-a975-0f17c954afc6\") "
Sep 17 17:03:42.141976 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.141848    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cfa55ac-7c33-4c71-a975-0f17c954afc6-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "2cfa55ac-7c33-4c71-a975-0f17c954afc6" (UID: "2cfa55ac-7c33-4c71-a975-0f17c954afc6"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:03:42.141976 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.141960    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cfa55ac-7c33-4c71-a975-0f17c954afc6-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "2cfa55ac-7c33-4c71-a975-0f17c954afc6" (UID: "2cfa55ac-7c33-4c71-a975-0f17c954afc6"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:03:42.142167 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.142143    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cfa55ac-7c33-4c71-a975-0f17c954afc6-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "2cfa55ac-7c33-4c71-a975-0f17c954afc6" (UID: "2cfa55ac-7c33-4c71-a975-0f17c954afc6"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:03:42.143839 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.143818    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2cfa55ac-7c33-4c71-a975-0f17c954afc6-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "2cfa55ac-7c33-4c71-a975-0f17c954afc6" (UID: "2cfa55ac-7c33-4c71-a975-0f17c954afc6"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:03:42.143921 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.143850    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cfa55ac-7c33-4c71-a975-0f17c954afc6-kube-api-access-pt9mn" (OuterVolumeSpecName: "kube-api-access-pt9mn") pod "2cfa55ac-7c33-4c71-a975-0f17c954afc6" (UID: "2cfa55ac-7c33-4c71-a975-0f17c954afc6"). InnerVolumeSpecName "kube-api-access-pt9mn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:03:42.242308 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.242271    2586 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2cfa55ac-7c33-4c71-a975-0f17c954afc6-proxy-tls\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 17:03:42.242308 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.242302    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pt9mn\" (UniqueName: \"kubernetes.io/projected/2cfa55ac-7c33-4c71-a975-0f17c954afc6-kube-api-access-pt9mn\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 17:03:42.242308 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.242312    2586 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2cfa55ac-7c33-4c71-a975-0f17c954afc6-cabundle-cert\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 17:03:42.242308 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.242320    2586 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2cfa55ac-7c33-4c71-a975-0f17c954afc6-kserve-provision-location\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 17:03:42.242583 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.242328    2586 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2cfa55ac-7c33-4c71-a975-0f17c954afc6-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 17:03:42.548060 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.548026    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2"]
Sep 17 17:03:42.548360 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.548347    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cfa55ac-7c33-4c71-a975-0f17c954afc6" containerName="kserve-container"
Sep 17 17:03:42.548406 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.548362    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cfa55ac-7c33-4c71-a975-0f17c954afc6" containerName="kserve-container"
Sep 17 17:03:42.548406 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.548380    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cfa55ac-7c33-4c71-a975-0f17c954afc6" containerName="storage-initializer"
Sep 17 17:03:42.548406 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.548385    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cfa55ac-7c33-4c71-a975-0f17c954afc6" containerName="storage-initializer"
Sep 17 17:03:42.548406 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.548398    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cfa55ac-7c33-4c71-a975-0f17c954afc6" containerName="kube-rbac-proxy"
Sep 17 17:03:42.548406 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.548403    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cfa55ac-7c33-4c71-a975-0f17c954afc6" containerName="kube-rbac-proxy"
Sep 17 17:03:42.548560 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.548457    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cfa55ac-7c33-4c71-a975-0f17c954afc6" containerName="kube-rbac-proxy"
Sep 17 17:03:42.548560 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.548464    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cfa55ac-7c33-4c71-a975-0f17c954afc6" containerName="kserve-container"
Sep 17 17:03:42.551583 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.551565    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2"
Sep 17 17:03:42.554288 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.554265    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-promote-kube-rbac-proxy-sar-config\""
Sep 17 17:03:42.554385 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.554266    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-promote-predictor-serving-cert\""
Sep 17 17:03:42.562988 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.562962    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2"]
Sep 17 17:03:42.645426 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.645390    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1724b53d-a9f6-4c61-9195-1a5987d84ead-cabundle-cert\") pod \"isvc-canary-promote-predictor-8bc5b857-gkln2\" (UID: \"1724b53d-a9f6-4c61-9195-1a5987d84ead\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2"
Sep 17 17:03:42.645426 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.645429    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1724b53d-a9f6-4c61-9195-1a5987d84ead-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-8bc5b857-gkln2\" (UID: \"1724b53d-a9f6-4c61-9195-1a5987d84ead\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2"
Sep 17 17:03:42.645668 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.645496    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4496j\" (UniqueName: \"kubernetes.io/projected/1724b53d-a9f6-4c61-9195-1a5987d84ead-kube-api-access-4496j\") pod \"isvc-canary-promote-predictor-8bc5b857-gkln2\" (UID: \"1724b53d-a9f6-4c61-9195-1a5987d84ead\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2"
Sep 17 17:03:42.645668 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.645543    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1724b53d-a9f6-4c61-9195-1a5987d84ead-proxy-tls\") pod \"isvc-canary-promote-predictor-8bc5b857-gkln2\" (UID: \"1724b53d-a9f6-4c61-9195-1a5987d84ead\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2"
Sep 17 17:03:42.645668 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.645572    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1724b53d-a9f6-4c61-9195-1a5987d84ead-kserve-provision-location\") pod \"isvc-canary-promote-predictor-8bc5b857-gkln2\" (UID: \"1724b53d-a9f6-4c61-9195-1a5987d84ead\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2"
Sep 17 17:03:42.746671 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.746621    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1724b53d-a9f6-4c61-9195-1a5987d84ead-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-8bc5b857-gkln2\" (UID: \"1724b53d-a9f6-4c61-9195-1a5987d84ead\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2"
Sep 17 17:03:42.746926 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.746704    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4496j\" (UniqueName: \"kubernetes.io/projected/1724b53d-a9f6-4c61-9195-1a5987d84ead-kube-api-access-4496j\") pod \"isvc-canary-promote-predictor-8bc5b857-gkln2\" (UID: \"1724b53d-a9f6-4c61-9195-1a5987d84ead\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2"
Sep 17 17:03:42.746926 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.746737    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1724b53d-a9f6-4c61-9195-1a5987d84ead-proxy-tls\") pod \"isvc-canary-promote-predictor-8bc5b857-gkln2\" (UID: \"1724b53d-a9f6-4c61-9195-1a5987d84ead\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2"
Sep 17 17:03:42.746926 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.746762    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1724b53d-a9f6-4c61-9195-1a5987d84ead-kserve-provision-location\") pod \"isvc-canary-promote-predictor-8bc5b857-gkln2\" (UID: \"1724b53d-a9f6-4c61-9195-1a5987d84ead\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2"
Sep 17 17:03:42.746926 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.746843    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1724b53d-a9f6-4c61-9195-1a5987d84ead-cabundle-cert\") pod \"isvc-canary-promote-predictor-8bc5b857-gkln2\" (UID: \"1724b53d-a9f6-4c61-9195-1a5987d84ead\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2"
Sep 17 17:03:42.747504 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.747456    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1724b53d-a9f6-4c61-9195-1a5987d84ead-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-8bc5b857-gkln2\" (UID: \"1724b53d-a9f6-4c61-9195-1a5987d84ead\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2"
Sep 17 17:03:42.747504 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.747477    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1724b53d-a9f6-4c61-9195-1a5987d84ead-kserve-provision-location\") pod \"isvc-canary-promote-predictor-8bc5b857-gkln2\" (UID: \"1724b53d-a9f6-4c61-9195-1a5987d84ead\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2"
Sep 17 17:03:42.747651 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.747514    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1724b53d-a9f6-4c61-9195-1a5987d84ead-cabundle-cert\") pod \"isvc-canary-promote-predictor-8bc5b857-gkln2\" (UID: \"1724b53d-a9f6-4c61-9195-1a5987d84ead\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2"
Sep 17 17:03:42.749769 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.749750    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1724b53d-a9f6-4c61-9195-1a5987d84ead-proxy-tls\") pod \"isvc-canary-promote-predictor-8bc5b857-gkln2\" (UID: \"1724b53d-a9f6-4c61-9195-1a5987d84ead\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2"
Sep 17 17:03:42.754612 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.754593    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4496j\" (UniqueName: \"kubernetes.io/projected/1724b53d-a9f6-4c61-9195-1a5987d84ead-kube-api-access-4496j\") pod \"isvc-canary-promote-predictor-8bc5b857-gkln2\" (UID: \"1724b53d-a9f6-4c61-9195-1a5987d84ead\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2"
Sep 17 17:03:42.861507 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.861426    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2"
Sep 17 17:03:42.928566 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.928532    2586 generic.go:356] "Generic (PLEG): container finished" podID="2cfa55ac-7c33-4c71-a975-0f17c954afc6" containerID="c00aa222d194301eda76426c0d2df92a8d17c32144f684a6f45b3c881dbf39f2" exitCode=0
Sep 17 17:03:42.928731 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.928601    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv"
Sep 17 17:03:42.928731 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.928605    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv" event={"ID":"2cfa55ac-7c33-4c71-a975-0f17c954afc6","Type":"ContainerDied","Data":"c00aa222d194301eda76426c0d2df92a8d17c32144f684a6f45b3c881dbf39f2"}
Sep 17 17:03:42.928731 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.928666    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv" event={"ID":"2cfa55ac-7c33-4c71-a975-0f17c954afc6","Type":"ContainerDied","Data":"04b395b3bfc11946afc8c56109ba5c7b6f5f10fbd4fe604aa568f4431059d36e"}
Sep 17 17:03:42.928731 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.928693    2586 scope.go:117] "RemoveContainer" containerID="90c82047e691380acae0f4b4f138aee12a29b114b3f98238b3388f529b63d905"
Sep 17 17:03:42.940120 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.940049    2586 scope.go:117] "RemoveContainer" containerID="c00aa222d194301eda76426c0d2df92a8d17c32144f684a6f45b3c881dbf39f2"
Sep 17 17:03:42.948351 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.948332    2586 scope.go:117] "RemoveContainer" containerID="317a94eeb6715a0e7f25a8e7fc7969c165d72bba1048d74159d0b3f9ecbe7480"
Sep 17 17:03:42.951325 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.951296    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv"]
Sep 17 17:03:42.955053 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.955033    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-create-predictor-8cb968449-cxzjv"]
Sep 17 17:03:42.956689 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.956672    2586 scope.go:117] "RemoveContainer" containerID="90c82047e691380acae0f4b4f138aee12a29b114b3f98238b3388f529b63d905"
Sep 17 17:03:42.957001 ip-10-0-130-106 kubenswrapper[2586]: E0917 17:03:42.956983    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90c82047e691380acae0f4b4f138aee12a29b114b3f98238b3388f529b63d905\": container with ID starting with 90c82047e691380acae0f4b4f138aee12a29b114b3f98238b3388f529b63d905 not found: ID does not exist" containerID="90c82047e691380acae0f4b4f138aee12a29b114b3f98238b3388f529b63d905"
Sep 17 17:03:42.957045 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.957008    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90c82047e691380acae0f4b4f138aee12a29b114b3f98238b3388f529b63d905"} err="failed to get container status \"90c82047e691380acae0f4b4f138aee12a29b114b3f98238b3388f529b63d905\": rpc error: code = NotFound desc = could not find container \"90c82047e691380acae0f4b4f138aee12a29b114b3f98238b3388f529b63d905\": container with ID starting with 90c82047e691380acae0f4b4f138aee12a29b114b3f98238b3388f529b63d905 not found: ID does not exist"
Sep 17 17:03:42.957045 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.957025    2586 scope.go:117] "RemoveContainer" containerID="c00aa222d194301eda76426c0d2df92a8d17c32144f684a6f45b3c881dbf39f2"
Sep 17 17:03:42.957264 ip-10-0-130-106 kubenswrapper[2586]: E0917 17:03:42.957246    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c00aa222d194301eda76426c0d2df92a8d17c32144f684a6f45b3c881dbf39f2\": container with ID starting with c00aa222d194301eda76426c0d2df92a8d17c32144f684a6f45b3c881dbf39f2 not found: ID does not exist" containerID="c00aa222d194301eda76426c0d2df92a8d17c32144f684a6f45b3c881dbf39f2"
Sep 17 17:03:42.957310 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.957268    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c00aa222d194301eda76426c0d2df92a8d17c32144f684a6f45b3c881dbf39f2"} err="failed to get container status \"c00aa222d194301eda76426c0d2df92a8d17c32144f684a6f45b3c881dbf39f2\": rpc error: code = NotFound desc = could not find container \"c00aa222d194301eda76426c0d2df92a8d17c32144f684a6f45b3c881dbf39f2\": container with ID starting with c00aa222d194301eda76426c0d2df92a8d17c32144f684a6f45b3c881dbf39f2 not found: ID does not exist"
Sep 17 17:03:42.957310 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.957280    2586 scope.go:117] "RemoveContainer" containerID="317a94eeb6715a0e7f25a8e7fc7969c165d72bba1048d74159d0b3f9ecbe7480"
Sep 17 17:03:42.957477 ip-10-0-130-106 kubenswrapper[2586]: E0917 17:03:42.957460    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"317a94eeb6715a0e7f25a8e7fc7969c165d72bba1048d74159d0b3f9ecbe7480\": container with ID starting with 317a94eeb6715a0e7f25a8e7fc7969c165d72bba1048d74159d0b3f9ecbe7480 not found: ID does not exist" containerID="317a94eeb6715a0e7f25a8e7fc7969c165d72bba1048d74159d0b3f9ecbe7480"
Sep 17 17:03:42.957515 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:42.957481    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"317a94eeb6715a0e7f25a8e7fc7969c165d72bba1048d74159d0b3f9ecbe7480"} err="failed to get container status \"317a94eeb6715a0e7f25a8e7fc7969c165d72bba1048d74159d0b3f9ecbe7480\": rpc error: code = NotFound desc = could not find container \"317a94eeb6715a0e7f25a8e7fc7969c165d72bba1048d74159d0b3f9ecbe7480\": container with ID starting with 317a94eeb6715a0e7f25a8e7fc7969c165d72bba1048d74159d0b3f9ecbe7480 not found: ID does not exist"
Sep 17 17:03:43.133378 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:43.133306    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2"]
Sep 17 17:03:43.135683 ip-10-0-130-106 kubenswrapper[2586]: W0917 17:03:43.135651    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1724b53d_a9f6_4c61_9195_1a5987d84ead.slice/crio-50ebaa52c82c03426bb595190676d88ece47ca7939909f84c89619e84cfca6d0 WatchSource:0}: Error finding container 50ebaa52c82c03426bb595190676d88ece47ca7939909f84c89619e84cfca6d0: Status 404 returned error can't find the container with id 50ebaa52c82c03426bb595190676d88ece47ca7939909f84c89619e84cfca6d0
Sep 17 17:03:43.326724 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:43.326684    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cfa55ac-7c33-4c71-a975-0f17c954afc6" path="/var/lib/kubelet/pods/2cfa55ac-7c33-4c71-a975-0f17c954afc6/volumes"
Sep 17 17:03:43.937217 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:43.937181    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2" event={"ID":"1724b53d-a9f6-4c61-9195-1a5987d84ead","Type":"ContainerStarted","Data":"8850364aabdccff74793e47ca8ded2d5bc7be6c88089588412d989cb925ff2b7"}
Sep 17 17:03:43.937420 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:43.937238    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2" event={"ID":"1724b53d-a9f6-4c61-9195-1a5987d84ead","Type":"ContainerStarted","Data":"50ebaa52c82c03426bb595190676d88ece47ca7939909f84c89619e84cfca6d0"}
Sep 17 17:03:47.962015 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:47.961983    2586 generic.go:356] "Generic (PLEG): container finished" podID="1724b53d-a9f6-4c61-9195-1a5987d84ead" containerID="8850364aabdccff74793e47ca8ded2d5bc7be6c88089588412d989cb925ff2b7" exitCode=0
Sep 17 17:03:47.962380 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:47.962062    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2" event={"ID":"1724b53d-a9f6-4c61-9195-1a5987d84ead","Type":"ContainerDied","Data":"8850364aabdccff74793e47ca8ded2d5bc7be6c88089588412d989cb925ff2b7"}
Sep 17 17:03:48.969060 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:48.969030    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2" event={"ID":"1724b53d-a9f6-4c61-9195-1a5987d84ead","Type":"ContainerStarted","Data":"757d1c7354e59d6160a244d0aef42ebe6bc165548ebcf600267cdf526e52e9f8"}
Sep 17 17:03:48.969424 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:48.969090    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2" event={"ID":"1724b53d-a9f6-4c61-9195-1a5987d84ead","Type":"ContainerStarted","Data":"72a36fb713ba7fe0ede36cc8fb54ce75691a9809a33fb62e0d46fd9edb042976"}
Sep 17 17:03:48.969424 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:48.969363    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2"
Sep 17 17:03:48.988557 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:48.988510    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2" podStartSLOduration=2.967486558 podStartE2EDuration="6.988495517s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.021008959s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:03:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:03:48.98748116 +0000 UTC m=+1299.688559821" watchObservedRunningTime="2026-09-17 17:03:48.988495517 +0000 UTC m=+1299.689574179"
Sep 17 17:03:49.973912 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:49.973862    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2"
Sep 17 17:03:49.975020 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:49.974989    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2" podUID="1724b53d-a9f6-4c61-9195-1a5987d84ead" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 17:03:50.984257 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:50.984217    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2" podUID="1724b53d-a9f6-4c61-9195-1a5987d84ead" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 17:03:55.989667 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:55.989638    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2"
Sep 17 17:03:55.990177 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:03:55.990148    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2" podUID="1724b53d-a9f6-4c61-9195-1a5987d84ead" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 17:04:05.991141 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:04:05.991098    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2" podUID="1724b53d-a9f6-4c61-9195-1a5987d84ead" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 17:04:15.991021 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:04:15.990982    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2" podUID="1724b53d-a9f6-4c61-9195-1a5987d84ead" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 17:04:25.990810 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:04:25.990721    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2" podUID="1724b53d-a9f6-4c61-9195-1a5987d84ead" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 17:04:35.991048 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:04:35.991005    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2" podUID="1724b53d-a9f6-4c61-9195-1a5987d84ead" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 17:04:45.990329 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:04:45.990287    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2" podUID="1724b53d-a9f6-4c61-9195-1a5987d84ead" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 17:04:55.990935 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:04:55.990891    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2" podUID="1724b53d-a9f6-4c61-9195-1a5987d84ead" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Sep 17 17:05:05.991833 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:05.991798    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2"
Sep 17 17:05:06.449658 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:06.449624    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2"]
Sep 17 17:05:06.449903 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:06.449852    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2" podUID="1724b53d-a9f6-4c61-9195-1a5987d84ead" containerName="kserve-container" containerID="cri-o://72a36fb713ba7fe0ede36cc8fb54ce75691a9809a33fb62e0d46fd9edb042976" gracePeriod=30
Sep 17 17:05:06.450036 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:06.449906    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2" podUID="1724b53d-a9f6-4c61-9195-1a5987d84ead" containerName="kube-rbac-proxy" containerID="cri-o://757d1c7354e59d6160a244d0aef42ebe6bc165548ebcf600267cdf526e52e9f8" gracePeriod=30
Sep 17 17:05:07.319249 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:07.319216    2586 generic.go:356] "Generic (PLEG): container finished" podID="1724b53d-a9f6-4c61-9195-1a5987d84ead" containerID="757d1c7354e59d6160a244d0aef42ebe6bc165548ebcf600267cdf526e52e9f8" exitCode=2
Sep 17 17:05:07.323429 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:07.323391    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2" event={"ID":"1724b53d-a9f6-4c61-9195-1a5987d84ead","Type":"ContainerDied","Data":"757d1c7354e59d6160a244d0aef42ebe6bc165548ebcf600267cdf526e52e9f8"}
Sep 17 17:05:10.985404 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:10.985358    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2" podUID="1724b53d-a9f6-4c61-9195-1a5987d84ead" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.37:8643/healthz\": dial tcp 10.133.0.37:8643: connect: connection refused"
Sep 17 17:05:11.200634 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.200609    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2"
Sep 17 17:05:11.288289 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.288248    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1724b53d-a9f6-4c61-9195-1a5987d84ead-kube-rbac-proxy-sar-config\") pod \"1724b53d-a9f6-4c61-9195-1a5987d84ead\" (UID: \"1724b53d-a9f6-4c61-9195-1a5987d84ead\") "
Sep 17 17:05:11.288289 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.288305    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4496j\" (UniqueName: \"kubernetes.io/projected/1724b53d-a9f6-4c61-9195-1a5987d84ead-kube-api-access-4496j\") pod \"1724b53d-a9f6-4c61-9195-1a5987d84ead\" (UID: \"1724b53d-a9f6-4c61-9195-1a5987d84ead\") "
Sep 17 17:05:11.288547 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.288336    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1724b53d-a9f6-4c61-9195-1a5987d84ead-cabundle-cert\") pod \"1724b53d-a9f6-4c61-9195-1a5987d84ead\" (UID: \"1724b53d-a9f6-4c61-9195-1a5987d84ead\") "
Sep 17 17:05:11.288547 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.288361    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1724b53d-a9f6-4c61-9195-1a5987d84ead-proxy-tls\") pod \"1724b53d-a9f6-4c61-9195-1a5987d84ead\" (UID: \"1724b53d-a9f6-4c61-9195-1a5987d84ead\") "
Sep 17 17:05:11.288547 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.288380    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1724b53d-a9f6-4c61-9195-1a5987d84ead-kserve-provision-location\") pod \"1724b53d-a9f6-4c61-9195-1a5987d84ead\" (UID: \"1724b53d-a9f6-4c61-9195-1a5987d84ead\") "
Sep 17 17:05:11.288705 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.288662    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1724b53d-a9f6-4c61-9195-1a5987d84ead-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "1724b53d-a9f6-4c61-9195-1a5987d84ead" (UID: "1724b53d-a9f6-4c61-9195-1a5987d84ead"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:05:11.288754 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.288728    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1724b53d-a9f6-4c61-9195-1a5987d84ead-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "1724b53d-a9f6-4c61-9195-1a5987d84ead" (UID: "1724b53d-a9f6-4c61-9195-1a5987d84ead"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:05:11.288754 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.288734    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1724b53d-a9f6-4c61-9195-1a5987d84ead-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "1724b53d-a9f6-4c61-9195-1a5987d84ead" (UID: "1724b53d-a9f6-4c61-9195-1a5987d84ead"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:05:11.290635 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.290613    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1724b53d-a9f6-4c61-9195-1a5987d84ead-kube-api-access-4496j" (OuterVolumeSpecName: "kube-api-access-4496j") pod "1724b53d-a9f6-4c61-9195-1a5987d84ead" (UID: "1724b53d-a9f6-4c61-9195-1a5987d84ead"). InnerVolumeSpecName "kube-api-access-4496j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:05:11.290690 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.290618    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1724b53d-a9f6-4c61-9195-1a5987d84ead-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "1724b53d-a9f6-4c61-9195-1a5987d84ead" (UID: "1724b53d-a9f6-4c61-9195-1a5987d84ead"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:05:11.337254 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.337228    2586 generic.go:356] "Generic (PLEG): container finished" podID="1724b53d-a9f6-4c61-9195-1a5987d84ead" containerID="72a36fb713ba7fe0ede36cc8fb54ce75691a9809a33fb62e0d46fd9edb042976" exitCode=0
Sep 17 17:05:11.337368 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.337265    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2" event={"ID":"1724b53d-a9f6-4c61-9195-1a5987d84ead","Type":"ContainerDied","Data":"72a36fb713ba7fe0ede36cc8fb54ce75691a9809a33fb62e0d46fd9edb042976"}
Sep 17 17:05:11.337368 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.337298    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2"
Sep 17 17:05:11.337368 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.337311    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2" event={"ID":"1724b53d-a9f6-4c61-9195-1a5987d84ead","Type":"ContainerDied","Data":"50ebaa52c82c03426bb595190676d88ece47ca7939909f84c89619e84cfca6d0"}
Sep 17 17:05:11.337368 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.337336    2586 scope.go:117] "RemoveContainer" containerID="757d1c7354e59d6160a244d0aef42ebe6bc165548ebcf600267cdf526e52e9f8"
Sep 17 17:05:11.346307 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.346291    2586 scope.go:117] "RemoveContainer" containerID="72a36fb713ba7fe0ede36cc8fb54ce75691a9809a33fb62e0d46fd9edb042976"
Sep 17 17:05:11.354856 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.354835    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2"]
Sep 17 17:05:11.356591 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.356530    2586 scope.go:117] "RemoveContainer" containerID="8850364aabdccff74793e47ca8ded2d5bc7be6c88089588412d989cb925ff2b7"
Sep 17 17:05:11.358764 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.358744    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-predictor-8bc5b857-gkln2"]
Sep 17 17:05:11.364143 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.364129    2586 scope.go:117] "RemoveContainer" containerID="757d1c7354e59d6160a244d0aef42ebe6bc165548ebcf600267cdf526e52e9f8"
Sep 17 17:05:11.364411 ip-10-0-130-106 kubenswrapper[2586]: E0917 17:05:11.364390    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"757d1c7354e59d6160a244d0aef42ebe6bc165548ebcf600267cdf526e52e9f8\": container with ID starting with 757d1c7354e59d6160a244d0aef42ebe6bc165548ebcf600267cdf526e52e9f8 not found: ID does not exist" containerID="757d1c7354e59d6160a244d0aef42ebe6bc165548ebcf600267cdf526e52e9f8"
Sep 17 17:05:11.364453 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.364419    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"757d1c7354e59d6160a244d0aef42ebe6bc165548ebcf600267cdf526e52e9f8"} err="failed to get container status \"757d1c7354e59d6160a244d0aef42ebe6bc165548ebcf600267cdf526e52e9f8\": rpc error: code = NotFound desc = could not find container \"757d1c7354e59d6160a244d0aef42ebe6bc165548ebcf600267cdf526e52e9f8\": container with ID starting with 757d1c7354e59d6160a244d0aef42ebe6bc165548ebcf600267cdf526e52e9f8 not found: ID does not exist"
Sep 17 17:05:11.364453 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.364436    2586 scope.go:117] "RemoveContainer" containerID="72a36fb713ba7fe0ede36cc8fb54ce75691a9809a33fb62e0d46fd9edb042976"
Sep 17 17:05:11.364675 ip-10-0-130-106 kubenswrapper[2586]: E0917 17:05:11.364658    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72a36fb713ba7fe0ede36cc8fb54ce75691a9809a33fb62e0d46fd9edb042976\": container with ID starting with 72a36fb713ba7fe0ede36cc8fb54ce75691a9809a33fb62e0d46fd9edb042976 not found: ID does not exist" containerID="72a36fb713ba7fe0ede36cc8fb54ce75691a9809a33fb62e0d46fd9edb042976"
Sep 17 17:05:11.364718 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.364684    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72a36fb713ba7fe0ede36cc8fb54ce75691a9809a33fb62e0d46fd9edb042976"} err="failed to get container status \"72a36fb713ba7fe0ede36cc8fb54ce75691a9809a33fb62e0d46fd9edb042976\": rpc error: code = NotFound desc = could not find container \"72a36fb713ba7fe0ede36cc8fb54ce75691a9809a33fb62e0d46fd9edb042976\": container with ID starting with 72a36fb713ba7fe0ede36cc8fb54ce75691a9809a33fb62e0d46fd9edb042976 not found: ID does not exist"
Sep 17 17:05:11.364718 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.364707    2586 scope.go:117] "RemoveContainer" containerID="8850364aabdccff74793e47ca8ded2d5bc7be6c88089588412d989cb925ff2b7"
Sep 17 17:05:11.364954 ip-10-0-130-106 kubenswrapper[2586]: E0917 17:05:11.364935    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8850364aabdccff74793e47ca8ded2d5bc7be6c88089588412d989cb925ff2b7\": container with ID starting with 8850364aabdccff74793e47ca8ded2d5bc7be6c88089588412d989cb925ff2b7 not found: ID does not exist" containerID="8850364aabdccff74793e47ca8ded2d5bc7be6c88089588412d989cb925ff2b7"
Sep 17 17:05:11.365040 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.364955    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8850364aabdccff74793e47ca8ded2d5bc7be6c88089588412d989cb925ff2b7"} err="failed to get container status \"8850364aabdccff74793e47ca8ded2d5bc7be6c88089588412d989cb925ff2b7\": rpc error: code = NotFound desc = could not find container \"8850364aabdccff74793e47ca8ded2d5bc7be6c88089588412d989cb925ff2b7\": container with ID starting with 8850364aabdccff74793e47ca8ded2d5bc7be6c88089588412d989cb925ff2b7 not found: ID does not exist"
Sep 17 17:05:11.388994 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.388973    2586 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1724b53d-a9f6-4c61-9195-1a5987d84ead-proxy-tls\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 17:05:11.388994 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.388997    2586 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1724b53d-a9f6-4c61-9195-1a5987d84ead-kserve-provision-location\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 17:05:11.389122 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.389006    2586 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1724b53d-a9f6-4c61-9195-1a5987d84ead-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 17:05:11.389122 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.389015    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4496j\" (UniqueName: \"kubernetes.io/projected/1724b53d-a9f6-4c61-9195-1a5987d84ead-kube-api-access-4496j\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 17:05:11.389122 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:11.389023    2586 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1724b53d-a9f6-4c61-9195-1a5987d84ead-cabundle-cert\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 17:05:12.623865 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.623834    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp"]
Sep 17 17:05:12.624251 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.624162    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1724b53d-a9f6-4c61-9195-1a5987d84ead" containerName="kserve-container"
Sep 17 17:05:12.624251 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.624175    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="1724b53d-a9f6-4c61-9195-1a5987d84ead" containerName="kserve-container"
Sep 17 17:05:12.624251 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.624197    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1724b53d-a9f6-4c61-9195-1a5987d84ead" containerName="storage-initializer"
Sep 17 17:05:12.624251 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.624203    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="1724b53d-a9f6-4c61-9195-1a5987d84ead" containerName="storage-initializer"
Sep 17 17:05:12.624251 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.624210    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1724b53d-a9f6-4c61-9195-1a5987d84ead" containerName="kube-rbac-proxy"
Sep 17 17:05:12.624251 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.624216    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="1724b53d-a9f6-4c61-9195-1a5987d84ead" containerName="kube-rbac-proxy"
Sep 17 17:05:12.624443 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.624264    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="1724b53d-a9f6-4c61-9195-1a5987d84ead" containerName="kube-rbac-proxy"
Sep 17 17:05:12.624443 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.624272    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="1724b53d-a9f6-4c61-9195-1a5987d84ead" containerName="kserve-container"
Sep 17 17:05:12.628541 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.628525    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp"
Sep 17 17:05:12.631159 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.631133    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Sep 17 17:05:12.631293 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.631195    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-promote-kube-rbac-proxy-sar-config\""
Sep 17 17:05:12.631447 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.631432    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 17 17:05:12.632134 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.632115    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 17 17:05:12.632134 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.632130    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-promote-v2-predictor-serving-cert\""
Sep 17 17:05:12.632273 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.632119    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Sep 17 17:05:12.632273 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.632123    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-tlgx8\""
Sep 17 17:05:12.637893 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.637851    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp"]
Sep 17 17:05:12.699050 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.699015    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cadd22d0-a34f-4940-a200-2f6d075b1c1c-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-65744c97b6-v7xcp\" (UID: \"cadd22d0-a34f-4940-a200-2f6d075b1c1c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp"
Sep 17 17:05:12.699211 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.699058    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cadd22d0-a34f-4940-a200-2f6d075b1c1c-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-65744c97b6-v7xcp\" (UID: \"cadd22d0-a34f-4940-a200-2f6d075b1c1c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp"
Sep 17 17:05:12.699211 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.699080    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/cadd22d0-a34f-4940-a200-2f6d075b1c1c-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-65744c97b6-v7xcp\" (UID: \"cadd22d0-a34f-4940-a200-2f6d075b1c1c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp"
Sep 17 17:05:12.699211 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.699122    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cadd22d0-a34f-4940-a200-2f6d075b1c1c-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-65744c97b6-v7xcp\" (UID: \"cadd22d0-a34f-4940-a200-2f6d075b1c1c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp"
Sep 17 17:05:12.699211 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.699164    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn9n5\" (UniqueName: \"kubernetes.io/projected/cadd22d0-a34f-4940-a200-2f6d075b1c1c-kube-api-access-gn9n5\") pod \"isvc-canary-promote-v2-predictor-65744c97b6-v7xcp\" (UID: \"cadd22d0-a34f-4940-a200-2f6d075b1c1c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp"
Sep 17 17:05:12.800101 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.800060    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn9n5\" (UniqueName: \"kubernetes.io/projected/cadd22d0-a34f-4940-a200-2f6d075b1c1c-kube-api-access-gn9n5\") pod \"isvc-canary-promote-v2-predictor-65744c97b6-v7xcp\" (UID: \"cadd22d0-a34f-4940-a200-2f6d075b1c1c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp"
Sep 17 17:05:12.800286 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.800129    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cadd22d0-a34f-4940-a200-2f6d075b1c1c-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-65744c97b6-v7xcp\" (UID: \"cadd22d0-a34f-4940-a200-2f6d075b1c1c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp"
Sep 17 17:05:12.800286 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.800154    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cadd22d0-a34f-4940-a200-2f6d075b1c1c-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-65744c97b6-v7xcp\" (UID: \"cadd22d0-a34f-4940-a200-2f6d075b1c1c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp"
Sep 17 17:05:12.800286 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.800174    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/cadd22d0-a34f-4940-a200-2f6d075b1c1c-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-65744c97b6-v7xcp\" (UID: \"cadd22d0-a34f-4940-a200-2f6d075b1c1c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp"
Sep 17 17:05:12.800286 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.800194    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cadd22d0-a34f-4940-a200-2f6d075b1c1c-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-65744c97b6-v7xcp\" (UID: \"cadd22d0-a34f-4940-a200-2f6d075b1c1c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp"
Sep 17 17:05:12.800604 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.800582    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cadd22d0-a34f-4940-a200-2f6d075b1c1c-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-65744c97b6-v7xcp\" (UID: \"cadd22d0-a34f-4940-a200-2f6d075b1c1c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp"
Sep 17 17:05:12.800775 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.800756    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cadd22d0-a34f-4940-a200-2f6d075b1c1c-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-65744c97b6-v7xcp\" (UID: \"cadd22d0-a34f-4940-a200-2f6d075b1c1c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp"
Sep 17 17:05:12.800820 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.800764    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/cadd22d0-a34f-4940-a200-2f6d075b1c1c-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-65744c97b6-v7xcp\" (UID: \"cadd22d0-a34f-4940-a200-2f6d075b1c1c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp"
Sep 17 17:05:12.802794 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.802775    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cadd22d0-a34f-4940-a200-2f6d075b1c1c-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-65744c97b6-v7xcp\" (UID: \"cadd22d0-a34f-4940-a200-2f6d075b1c1c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp"
Sep 17 17:05:12.808735 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.808715    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn9n5\" (UniqueName: \"kubernetes.io/projected/cadd22d0-a34f-4940-a200-2f6d075b1c1c-kube-api-access-gn9n5\") pod \"isvc-canary-promote-v2-predictor-65744c97b6-v7xcp\" (UID: \"cadd22d0-a34f-4940-a200-2f6d075b1c1c\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp"
Sep 17 17:05:12.939119 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:12.939024    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp"
Sep 17 17:05:13.212978 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:13.212944    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp"]
Sep 17 17:05:13.214565 ip-10-0-130-106 kubenswrapper[2586]: W0917 17:05:13.214539    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcadd22d0_a34f_4940_a200_2f6d075b1c1c.slice/crio-0c6580ab5593453ccc7cd96dbb645dcb42e4029099f884cf854edcedb596a59e WatchSource:0}: Error finding container 0c6580ab5593453ccc7cd96dbb645dcb42e4029099f884cf854edcedb596a59e: Status 404 returned error can't find the container with id 0c6580ab5593453ccc7cd96dbb645dcb42e4029099f884cf854edcedb596a59e
Sep 17 17:05:13.324274 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:13.324239    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1724b53d-a9f6-4c61-9195-1a5987d84ead" path="/var/lib/kubelet/pods/1724b53d-a9f6-4c61-9195-1a5987d84ead/volumes"
Sep 17 17:05:13.349160 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:13.349127    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp" event={"ID":"cadd22d0-a34f-4940-a200-2f6d075b1c1c","Type":"ContainerStarted","Data":"2dc50eb115b8650feac778d6f1a122519f7620e649e647d2e3375499606c2b02"}
Sep 17 17:05:13.349294 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:13.349184    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp" event={"ID":"cadd22d0-a34f-4940-a200-2f6d075b1c1c","Type":"ContainerStarted","Data":"0c6580ab5593453ccc7cd96dbb645dcb42e4029099f884cf854edcedb596a59e"}
Sep 17 17:05:18.370451 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:18.370416    2586 generic.go:356] "Generic (PLEG): container finished" podID="cadd22d0-a34f-4940-a200-2f6d075b1c1c" containerID="2dc50eb115b8650feac778d6f1a122519f7620e649e647d2e3375499606c2b02" exitCode=0
Sep 17 17:05:18.370842 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:18.370461    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp" event={"ID":"cadd22d0-a34f-4940-a200-2f6d075b1c1c","Type":"ContainerDied","Data":"2dc50eb115b8650feac778d6f1a122519f7620e649e647d2e3375499606c2b02"}
Sep 17 17:05:19.376515 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:19.376481    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp" event={"ID":"cadd22d0-a34f-4940-a200-2f6d075b1c1c","Type":"ContainerStarted","Data":"eef1ba9db4e7ff220da001a2d66c6fbf3fc7ee254c5d8cdba48a2ae54721b478"}
Sep 17 17:05:19.376924 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:19.376542    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp" event={"ID":"cadd22d0-a34f-4940-a200-2f6d075b1c1c","Type":"ContainerStarted","Data":"a3f79cb6e5a00470ca4755374c7c5437b6c0708d20f8e01a5f11e5a939438725"}
Sep 17 17:05:19.376924 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:19.376851    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp"
Sep 17 17:05:19.377025 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:19.376992    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp"
Sep 17 17:05:19.378405 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:19.378377    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp" podUID="cadd22d0-a34f-4940-a200-2f6d075b1c1c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 17:05:19.395683 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:19.395632    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp" podStartSLOduration=3.300875268 podStartE2EDuration="7.395615226s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.094739958s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:05:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:05:19.393811243 +0000 UTC m=+1390.094889905" watchObservedRunningTime="2026-09-17 17:05:19.395615226 +0000 UTC m=+1390.096693887"
Sep 17 17:05:20.380810 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:20.380768    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp" podUID="cadd22d0-a34f-4940-a200-2f6d075b1c1c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 17:05:25.385208 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:25.385178    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp"
Sep 17 17:05:25.385726 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:25.385698    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp" podUID="cadd22d0-a34f-4940-a200-2f6d075b1c1c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 17:05:35.386307 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:35.386265    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp" podUID="cadd22d0-a34f-4940-a200-2f6d075b1c1c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 17:05:45.386379 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:45.386341    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp" podUID="cadd22d0-a34f-4940-a200-2f6d075b1c1c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 17:05:55.385907 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:05:55.385805    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp" podUID="cadd22d0-a34f-4940-a200-2f6d075b1c1c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 17:06:05.386054 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:05.386015    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp" podUID="cadd22d0-a34f-4940-a200-2f6d075b1c1c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 17:06:15.386055 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:15.386015    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp" podUID="cadd22d0-a34f-4940-a200-2f6d075b1c1c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 17:06:25.386210 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:25.386170    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp" podUID="cadd22d0-a34f-4940-a200-2f6d075b1c1c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 17:06:35.386897 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:35.386848    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp"
Sep 17 17:06:42.765731 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:42.765698    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp"]
Sep 17 17:06:42.766204 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:42.765951    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp" podUID="cadd22d0-a34f-4940-a200-2f6d075b1c1c" containerName="kserve-container" containerID="cri-o://a3f79cb6e5a00470ca4755374c7c5437b6c0708d20f8e01a5f11e5a939438725" gracePeriod=30
Sep 17 17:06:42.766204 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:42.766006    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp" podUID="cadd22d0-a34f-4940-a200-2f6d075b1c1c" containerName="kube-rbac-proxy" containerID="cri-o://eef1ba9db4e7ff220da001a2d66c6fbf3fc7ee254c5d8cdba48a2ae54721b478" gracePeriod=30
Sep 17 17:06:43.733610 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:43.733572    2586 generic.go:356] "Generic (PLEG): container finished" podID="cadd22d0-a34f-4940-a200-2f6d075b1c1c" containerID="eef1ba9db4e7ff220da001a2d66c6fbf3fc7ee254c5d8cdba48a2ae54721b478" exitCode=2
Sep 17 17:06:43.733793 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:43.733647    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp" event={"ID":"cadd22d0-a34f-4940-a200-2f6d075b1c1c","Type":"ContainerDied","Data":"eef1ba9db4e7ff220da001a2d66c6fbf3fc7ee254c5d8cdba48a2ae54721b478"}
Sep 17 17:06:45.381581 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:45.381541    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp" podUID="cadd22d0-a34f-4940-a200-2f6d075b1c1c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.38:8643/healthz\": dial tcp 10.133.0.38:8643: connect: connection refused"
Sep 17 17:06:45.385925 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:45.385896    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp" podUID="cadd22d0-a34f-4940-a200-2f6d075b1c1c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Sep 17 17:06:47.405819 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.405797    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp"
Sep 17 17:06:47.499089 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.499057    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cadd22d0-a34f-4940-a200-2f6d075b1c1c-proxy-tls\") pod \"cadd22d0-a34f-4940-a200-2f6d075b1c1c\" (UID: \"cadd22d0-a34f-4940-a200-2f6d075b1c1c\") "
Sep 17 17:06:47.499266 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.499098    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cadd22d0-a34f-4940-a200-2f6d075b1c1c-kube-rbac-proxy-sar-config\") pod \"cadd22d0-a34f-4940-a200-2f6d075b1c1c\" (UID: \"cadd22d0-a34f-4940-a200-2f6d075b1c1c\") "
Sep 17 17:06:47.499266 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.499124    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gn9n5\" (UniqueName: \"kubernetes.io/projected/cadd22d0-a34f-4940-a200-2f6d075b1c1c-kube-api-access-gn9n5\") pod \"cadd22d0-a34f-4940-a200-2f6d075b1c1c\" (UID: \"cadd22d0-a34f-4940-a200-2f6d075b1c1c\") "
Sep 17 17:06:47.499266 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.499153    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/cadd22d0-a34f-4940-a200-2f6d075b1c1c-cabundle-cert\") pod \"cadd22d0-a34f-4940-a200-2f6d075b1c1c\" (UID: \"cadd22d0-a34f-4940-a200-2f6d075b1c1c\") "
Sep 17 17:06:47.499401 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.499294    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cadd22d0-a34f-4940-a200-2f6d075b1c1c-kserve-provision-location\") pod \"cadd22d0-a34f-4940-a200-2f6d075b1c1c\" (UID: \"cadd22d0-a34f-4940-a200-2f6d075b1c1c\") "
Sep 17 17:06:47.499582 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.499538    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cadd22d0-a34f-4940-a200-2f6d075b1c1c-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "cadd22d0-a34f-4940-a200-2f6d075b1c1c" (UID: "cadd22d0-a34f-4940-a200-2f6d075b1c1c"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:06:47.499701 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.499579    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cadd22d0-a34f-4940-a200-2f6d075b1c1c-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "cadd22d0-a34f-4940-a200-2f6d075b1c1c" (UID: "cadd22d0-a34f-4940-a200-2f6d075b1c1c"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:06:47.499701 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.499591    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cadd22d0-a34f-4940-a200-2f6d075b1c1c-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "cadd22d0-a34f-4940-a200-2f6d075b1c1c" (UID: "cadd22d0-a34f-4940-a200-2f6d075b1c1c"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:06:47.501471 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.501425    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cadd22d0-a34f-4940-a200-2f6d075b1c1c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "cadd22d0-a34f-4940-a200-2f6d075b1c1c" (UID: "cadd22d0-a34f-4940-a200-2f6d075b1c1c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:06:47.501471 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.501437    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cadd22d0-a34f-4940-a200-2f6d075b1c1c-kube-api-access-gn9n5" (OuterVolumeSpecName: "kube-api-access-gn9n5") pod "cadd22d0-a34f-4940-a200-2f6d075b1c1c" (UID: "cadd22d0-a34f-4940-a200-2f6d075b1c1c"). InnerVolumeSpecName "kube-api-access-gn9n5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:06:47.600351 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.600308    2586 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cadd22d0-a34f-4940-a200-2f6d075b1c1c-proxy-tls\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 17:06:47.600351 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.600343    2586 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cadd22d0-a34f-4940-a200-2f6d075b1c1c-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 17:06:47.600351 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.600353    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gn9n5\" (UniqueName: \"kubernetes.io/projected/cadd22d0-a34f-4940-a200-2f6d075b1c1c-kube-api-access-gn9n5\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 17:06:47.600351 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.600364    2586 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/cadd22d0-a34f-4940-a200-2f6d075b1c1c-cabundle-cert\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 17:06:47.600609 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.600374    2586 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cadd22d0-a34f-4940-a200-2f6d075b1c1c-kserve-provision-location\") on node \"ip-10-0-130-106.ec2.internal\" DevicePath \"\""
Sep 17 17:06:47.752862 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.752766    2586 generic.go:356] "Generic (PLEG): container finished" podID="cadd22d0-a34f-4940-a200-2f6d075b1c1c" containerID="a3f79cb6e5a00470ca4755374c7c5437b6c0708d20f8e01a5f11e5a939438725" exitCode=0
Sep 17 17:06:47.752862 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.752822    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp"
Sep 17 17:06:47.753076 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.752860    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp" event={"ID":"cadd22d0-a34f-4940-a200-2f6d075b1c1c","Type":"ContainerDied","Data":"a3f79cb6e5a00470ca4755374c7c5437b6c0708d20f8e01a5f11e5a939438725"}
Sep 17 17:06:47.753076 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.752937    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp" event={"ID":"cadd22d0-a34f-4940-a200-2f6d075b1c1c","Type":"ContainerDied","Data":"0c6580ab5593453ccc7cd96dbb645dcb42e4029099f884cf854edcedb596a59e"}
Sep 17 17:06:47.753076 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.752964    2586 scope.go:117] "RemoveContainer" containerID="eef1ba9db4e7ff220da001a2d66c6fbf3fc7ee254c5d8cdba48a2ae54721b478"
Sep 17 17:06:47.761479 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.761461    2586 scope.go:117] "RemoveContainer" containerID="a3f79cb6e5a00470ca4755374c7c5437b6c0708d20f8e01a5f11e5a939438725"
Sep 17 17:06:47.769071 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.769050    2586 scope.go:117] "RemoveContainer" containerID="2dc50eb115b8650feac778d6f1a122519f7620e649e647d2e3375499606c2b02"
Sep 17 17:06:47.774152 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.774127    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp"]
Sep 17 17:06:47.776486 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.776471    2586 scope.go:117] "RemoveContainer" containerID="eef1ba9db4e7ff220da001a2d66c6fbf3fc7ee254c5d8cdba48a2ae54721b478"
Sep 17 17:06:47.776731 ip-10-0-130-106 kubenswrapper[2586]: E0917 17:06:47.776714    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eef1ba9db4e7ff220da001a2d66c6fbf3fc7ee254c5d8cdba48a2ae54721b478\": container with ID starting with eef1ba9db4e7ff220da001a2d66c6fbf3fc7ee254c5d8cdba48a2ae54721b478 not found: ID does not exist" containerID="eef1ba9db4e7ff220da001a2d66c6fbf3fc7ee254c5d8cdba48a2ae54721b478"
Sep 17 17:06:47.776791 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.776736    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eef1ba9db4e7ff220da001a2d66c6fbf3fc7ee254c5d8cdba48a2ae54721b478"} err="failed to get container status \"eef1ba9db4e7ff220da001a2d66c6fbf3fc7ee254c5d8cdba48a2ae54721b478\": rpc error: code = NotFound desc = could not find container \"eef1ba9db4e7ff220da001a2d66c6fbf3fc7ee254c5d8cdba48a2ae54721b478\": container with ID starting with eef1ba9db4e7ff220da001a2d66c6fbf3fc7ee254c5d8cdba48a2ae54721b478 not found: ID does not exist"
Sep 17 17:06:47.776791 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.776754    2586 scope.go:117] "RemoveContainer" containerID="a3f79cb6e5a00470ca4755374c7c5437b6c0708d20f8e01a5f11e5a939438725"
Sep 17 17:06:47.777005 ip-10-0-130-106 kubenswrapper[2586]: E0917 17:06:47.776990    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3f79cb6e5a00470ca4755374c7c5437b6c0708d20f8e01a5f11e5a939438725\": container with ID starting with a3f79cb6e5a00470ca4755374c7c5437b6c0708d20f8e01a5f11e5a939438725 not found: ID does not exist" containerID="a3f79cb6e5a00470ca4755374c7c5437b6c0708d20f8e01a5f11e5a939438725"
Sep 17 17:06:47.777054 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.777009    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3f79cb6e5a00470ca4755374c7c5437b6c0708d20f8e01a5f11e5a939438725"} err="failed to get container status \"a3f79cb6e5a00470ca4755374c7c5437b6c0708d20f8e01a5f11e5a939438725\": rpc error: code = NotFound desc = could not find container \"a3f79cb6e5a00470ca4755374c7c5437b6c0708d20f8e01a5f11e5a939438725\": container with ID starting with a3f79cb6e5a00470ca4755374c7c5437b6c0708d20f8e01a5f11e5a939438725 not found: ID does not exist"
Sep 17 17:06:47.777054 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.777019    2586 scope.go:117] "RemoveContainer" containerID="2dc50eb115b8650feac778d6f1a122519f7620e649e647d2e3375499606c2b02"
Sep 17 17:06:47.777258 ip-10-0-130-106 kubenswrapper[2586]: E0917 17:06:47.777242    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dc50eb115b8650feac778d6f1a122519f7620e649e647d2e3375499606c2b02\": container with ID starting with 2dc50eb115b8650feac778d6f1a122519f7620e649e647d2e3375499606c2b02 not found: ID does not exist" containerID="2dc50eb115b8650feac778d6f1a122519f7620e649e647d2e3375499606c2b02"
Sep 17 17:06:47.777308 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.777263    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dc50eb115b8650feac778d6f1a122519f7620e649e647d2e3375499606c2b02"} err="failed to get container status \"2dc50eb115b8650feac778d6f1a122519f7620e649e647d2e3375499606c2b02\": rpc error: code = NotFound desc = could not find container \"2dc50eb115b8650feac778d6f1a122519f7620e649e647d2e3375499606c2b02\": container with ID starting with 2dc50eb115b8650feac778d6f1a122519f7620e649e647d2e3375499606c2b02 not found: ID does not exist"
Sep 17 17:06:47.779617 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:47.779596    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-65744c97b6-v7xcp"]
Sep 17 17:06:49.324561 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:06:49.324524    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cadd22d0-a34f-4940-a200-2f6d075b1c1c" path="/var/lib/kubelet/pods/cadd22d0-a34f-4940-a200-2f6d075b1c1c/volumes"
Sep 17 17:07:12.989285 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:07:12.989184    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22fdn_68559fca-75ed-4ee8-bedd-57b04fbd3f4d/ovn-acl-logging/0.log"
Sep 17 17:07:12.992475 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:07:12.990141    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22fdn_68559fca-75ed-4ee8-bedd-57b04fbd3f4d/ovn-acl-logging/0.log"
Sep 17 17:09:26.231007 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:26.230938    2586 ???:1] "http: TLS handshake error from 10.0.138.117:43840: EOF"
Sep 17 17:09:26.237179 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:26.237159    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jpzxf_36250538-b6ed-4b8d-8788-a205a4dd53c6/global-pull-secret-syncer/0.log"
Sep 17 17:09:26.412992 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:26.412962    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-h8gmd_641beacd-a18d-42fe-a1f6-e75256b15892/konnectivity-agent/0.log"
Sep 17 17:09:26.433852 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:26.433826    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-106.ec2.internal_0052a71b7d9061ab5e61bebaf109be27/haproxy/0.log"
Sep 17 17:09:30.569943 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:30.569913    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3/alertmanager/0.log"
Sep 17 17:09:30.594329 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:30.594299    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3/config-reloader/0.log"
Sep 17 17:09:30.621919 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:30.621893    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3/kube-rbac-proxy-web/0.log"
Sep 17 17:09:30.692735 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:30.692709    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3/kube-rbac-proxy/0.log"
Sep 17 17:09:30.719209 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:30.719182    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3/kube-rbac-proxy-metric/0.log"
Sep 17 17:09:30.745829 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:30.745801    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3/prom-label-proxy/0.log"
Sep 17 17:09:30.770728 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:30.770707    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_df2f7bfc-ac59-46ec-8e2a-bbef2de4b6e3/init-config-reloader/0.log"
Sep 17 17:09:30.980706 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:30.980617    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2k2gh_ae9d0d08-59de-454d-b7c5-45fc0393a7f3/node-exporter/0.log"
Sep 17 17:09:31.000744 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:31.000718    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2k2gh_ae9d0d08-59de-454d-b7c5-45fc0393a7f3/kube-rbac-proxy/0.log"
Sep 17 17:09:31.020571 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:31.020551    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2k2gh_ae9d0d08-59de-454d-b7c5-45fc0393a7f3/init-textfile/0.log"
Sep 17 17:09:31.467118 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:31.467087    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-fzbz8_93e836ed-cf52-4673-8258-ba74220df38b/prometheus-operator/0.log"
Sep 17 17:09:31.486639 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:31.486618    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-fzbz8_93e836ed-cf52-4673-8258-ba74220df38b/kube-rbac-proxy/0.log"
Sep 17 17:09:31.542403 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:31.542376    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-869c579c6-cclth_22bc52f8-271f-4582-af20-40590e7f9935/telemeter-client/0.log"
Sep 17 17:09:31.566968 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:31.566950    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-869c579c6-cclth_22bc52f8-271f-4582-af20-40590e7f9935/reload/0.log"
Sep 17 17:09:31.585916 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:31.585895    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-869c579c6-cclth_22bc52f8-271f-4582-af20-40590e7f9935/kube-rbac-proxy/0.log"
Sep 17 17:09:33.980392 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:33.980357    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-79cb5fcc48-96qkf_25e1699a-db36-41fa-8ab9-03b02e86d1cb/console/0.log"
Sep 17 17:09:34.221646 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.221615    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xngqr/perf-node-gather-daemonset-c7sfp"]
Sep 17 17:09:34.221953 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.221941    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cadd22d0-a34f-4940-a200-2f6d075b1c1c" containerName="kube-rbac-proxy"
Sep 17 17:09:34.222022 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.221954    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="cadd22d0-a34f-4940-a200-2f6d075b1c1c" containerName="kube-rbac-proxy"
Sep 17 17:09:34.222022 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.221972    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cadd22d0-a34f-4940-a200-2f6d075b1c1c" containerName="storage-initializer"
Sep 17 17:09:34.222022 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.221977    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="cadd22d0-a34f-4940-a200-2f6d075b1c1c" containerName="storage-initializer"
Sep 17 17:09:34.222022 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.221989    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cadd22d0-a34f-4940-a200-2f6d075b1c1c" containerName="kserve-container"
Sep 17 17:09:34.222022 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.221994    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="cadd22d0-a34f-4940-a200-2f6d075b1c1c" containerName="kserve-container"
Sep 17 17:09:34.222185 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.222039    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="cadd22d0-a34f-4940-a200-2f6d075b1c1c" containerName="kube-rbac-proxy"
Sep 17 17:09:34.222185 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.222049    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="cadd22d0-a34f-4940-a200-2f6d075b1c1c" containerName="kserve-container"
Sep 17 17:09:34.225153 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.225134    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xngqr/perf-node-gather-daemonset-c7sfp"
Sep 17 17:09:34.227681 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.227665    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xngqr\"/\"kube-root-ca.crt\""
Sep 17 17:09:34.227751 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.227667    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xngqr\"/\"default-dockercfg-zpvgc\""
Sep 17 17:09:34.228782 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.228766    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xngqr\"/\"openshift-service-ca.crt\""
Sep 17 17:09:34.233759 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.233737    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xngqr/perf-node-gather-daemonset-c7sfp"]
Sep 17 17:09:34.341212 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.341174    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b-lib-modules\") pod \"perf-node-gather-daemonset-c7sfp\" (UID: \"96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b\") " pod="openshift-must-gather-xngqr/perf-node-gather-daemonset-c7sfp"
Sep 17 17:09:34.341382 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.341226    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b-proc\") pod \"perf-node-gather-daemonset-c7sfp\" (UID: \"96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b\") " pod="openshift-must-gather-xngqr/perf-node-gather-daemonset-c7sfp"
Sep 17 17:09:34.341382 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.341266    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b-sys\") pod \"perf-node-gather-daemonset-c7sfp\" (UID: \"96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b\") " pod="openshift-must-gather-xngqr/perf-node-gather-daemonset-c7sfp"
Sep 17 17:09:34.341382 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.341323    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqb5z\" (UniqueName: \"kubernetes.io/projected/96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b-kube-api-access-lqb5z\") pod \"perf-node-gather-daemonset-c7sfp\" (UID: \"96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b\") " pod="openshift-must-gather-xngqr/perf-node-gather-daemonset-c7sfp"
Sep 17 17:09:34.341382 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.341366    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b-podres\") pod \"perf-node-gather-daemonset-c7sfp\" (UID: \"96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b\") " pod="openshift-must-gather-xngqr/perf-node-gather-daemonset-c7sfp"
Sep 17 17:09:34.442130 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.442087    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b-lib-modules\") pod \"perf-node-gather-daemonset-c7sfp\" (UID: \"96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b\") " pod="openshift-must-gather-xngqr/perf-node-gather-daemonset-c7sfp"
Sep 17 17:09:34.442130 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.442124    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b-proc\") pod \"perf-node-gather-daemonset-c7sfp\" (UID: \"96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b\") " pod="openshift-must-gather-xngqr/perf-node-gather-daemonset-c7sfp"
Sep 17 17:09:34.442396 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.442143    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b-sys\") pod \"perf-node-gather-daemonset-c7sfp\" (UID: \"96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b\") " pod="openshift-must-gather-xngqr/perf-node-gather-daemonset-c7sfp"
Sep 17 17:09:34.442396 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.442179    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqb5z\" (UniqueName: \"kubernetes.io/projected/96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b-kube-api-access-lqb5z\") pod \"perf-node-gather-daemonset-c7sfp\" (UID: \"96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b\") " pod="openshift-must-gather-xngqr/perf-node-gather-daemonset-c7sfp"
Sep 17 17:09:34.442396 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.442204    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b-podres\") pod \"perf-node-gather-daemonset-c7sfp\" (UID: \"96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b\") " pod="openshift-must-gather-xngqr/perf-node-gather-daemonset-c7sfp"
Sep 17 17:09:34.442396 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.442246    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b-proc\") pod \"perf-node-gather-daemonset-c7sfp\" (UID: \"96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b\") " pod="openshift-must-gather-xngqr/perf-node-gather-daemonset-c7sfp"
Sep 17 17:09:34.442396 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.442268    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b-lib-modules\") pod \"perf-node-gather-daemonset-c7sfp\" (UID: \"96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b\") " pod="openshift-must-gather-xngqr/perf-node-gather-daemonset-c7sfp"
Sep 17 17:09:34.442396 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.442288    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b-sys\") pod \"perf-node-gather-daemonset-c7sfp\" (UID: \"96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b\") " pod="openshift-must-gather-xngqr/perf-node-gather-daemonset-c7sfp"
Sep 17 17:09:34.442396 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.442319    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b-podres\") pod \"perf-node-gather-daemonset-c7sfp\" (UID: \"96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b\") " pod="openshift-must-gather-xngqr/perf-node-gather-daemonset-c7sfp"
Sep 17 17:09:34.450261 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.450236    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqb5z\" (UniqueName: \"kubernetes.io/projected/96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b-kube-api-access-lqb5z\") pod \"perf-node-gather-daemonset-c7sfp\" (UID: \"96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b\") " pod="openshift-must-gather-xngqr/perf-node-gather-daemonset-c7sfp"
Sep 17 17:09:34.535360 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.535335    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xngqr/perf-node-gather-daemonset-c7sfp"
Sep 17 17:09:34.805005 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:34.804925    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xngqr/perf-node-gather-daemonset-c7sfp"]
Sep 17 17:09:34.808182 ip-10-0-130-106 kubenswrapper[2586]: W0917 17:09:34.808154    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod96652ff2_b5ba_49d8_b6e3_6ee6d7b4ff4b.slice/crio-883255062b4499b98daa4cacbd093a1892a921918b1cde19dbfa315ce166f6c7 WatchSource:0}: Error finding container 883255062b4499b98daa4cacbd093a1892a921918b1cde19dbfa315ce166f6c7: Status 404 returned error can't find the container with id 883255062b4499b98daa4cacbd093a1892a921918b1cde19dbfa315ce166f6c7
Sep 17 17:09:35.464905 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:35.464853    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xngqr/perf-node-gather-daemonset-c7sfp" event={"ID":"96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b","Type":"ContainerStarted","Data":"251cd4898916c3f9d60447a6618c3c606f4d2f5191975196b9f6fb9ad440dcef"}
Sep 17 17:09:35.465310 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:35.464921    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xngqr/perf-node-gather-daemonset-c7sfp" event={"ID":"96652ff2-b5ba-49d8-b6e3-6ee6d7b4ff4b","Type":"ContainerStarted","Data":"883255062b4499b98daa4cacbd093a1892a921918b1cde19dbfa315ce166f6c7"}
Sep 17 17:09:35.465310 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:35.464957    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xngqr/perf-node-gather-daemonset-c7sfp"
Sep 17 17:09:35.483227 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:35.483185    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xngqr/perf-node-gather-daemonset-c7sfp" podStartSLOduration=1.483171283 podStartE2EDuration="1.483171283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:09:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:09:35.482427199 +0000 UTC m=+1646.183505861" watchObservedRunningTime="2026-09-17 17:09:35.483171283 +0000 UTC m=+1646.184249944"
Sep 17 17:09:35.513514 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:35.513486    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pnvqx_5976328d-6d8e-4fcd-a23b-c1f229f2713f/dns/0.log"
Sep 17 17:09:35.531922 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:35.531897    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pnvqx_5976328d-6d8e-4fcd-a23b-c1f229f2713f/kube-rbac-proxy/0.log"
Sep 17 17:09:35.595811 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:35.595787    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vsjlx_7615eda5-5be1-4711-a3d6-e743d87c267a/dns-node-resolver/0.log"
Sep 17 17:09:36.051374 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:36.051345    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6b849696d6-2lzx6_e5484fbc-acb2-4ac6-ad26-0c7170c87ae5/registry/0.log"
Sep 17 17:09:36.095816 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:36.095792    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fktbq_0fc32be3-c783-4e1b-83bc-45d67217b049/node-ca/0.log"
Sep 17 17:09:37.215271 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:37.215223    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wn8js_38641319-d38c-4818-a3dd-538d20dec869/serve-healthcheck-canary/0.log"
Sep 17 17:09:37.697804 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:37.697776    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zf89n_ae5edc41-0024-4ff8-825f-3fc6e3d20d08/kube-rbac-proxy/0.log"
Sep 17 17:09:37.716357 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:37.716327    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zf89n_ae5edc41-0024-4ff8-825f-3fc6e3d20d08/exporter/0.log"
Sep 17 17:09:37.734998 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:37.734973    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zf89n_ae5edc41-0024-4ff8-825f-3fc6e3d20d08/extractor/0.log"
Sep 17 17:09:40.006326 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:40.006288    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-846cf5d849-g7m2q_40a49f5a-e83c-4853-ae1a-4d3bb6904487/manager/0.log"
Sep 17 17:09:40.057941 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:40.057908    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-645f9c8fd6-p5dqp_ddcd98b1-18b4-42eb-b229-1423d2740ecd/manager/0.log"
Sep 17 17:09:40.106919 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:40.106867    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-7b954ffc95-nxrcc_ff684b6d-4906-4d60-89ae-ae2db177413a/server/0.log"
Sep 17 17:09:41.479594 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:41.479567    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xngqr/perf-node-gather-daemonset-c7sfp"
Sep 17 17:09:45.839299 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:45.839271    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cbv5m_b440e323-9b5d-4e31-b6e5-cd6c68d66451/kube-multus-additional-cni-plugins/0.log"
Sep 17 17:09:45.860761 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:45.860738    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cbv5m_b440e323-9b5d-4e31-b6e5-cd6c68d66451/egress-router-binary-copy/0.log"
Sep 17 17:09:45.879524 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:45.879499    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cbv5m_b440e323-9b5d-4e31-b6e5-cd6c68d66451/cni-plugins/0.log"
Sep 17 17:09:45.898075 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:45.898054    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cbv5m_b440e323-9b5d-4e31-b6e5-cd6c68d66451/bond-cni-plugin/0.log"
Sep 17 17:09:45.917586 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:45.917565    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cbv5m_b440e323-9b5d-4e31-b6e5-cd6c68d66451/routeoverride-cni/0.log"
Sep 17 17:09:45.936391 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:45.936369    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cbv5m_b440e323-9b5d-4e31-b6e5-cd6c68d66451/whereabouts-cni-bincopy/0.log"
Sep 17 17:09:45.955447 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:45.955423    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cbv5m_b440e323-9b5d-4e31-b6e5-cd6c68d66451/whereabouts-cni/0.log"
Sep 17 17:09:46.288791 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:46.288768    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-hw7f7_b9a0d1a1-6538-4d4f-be6a-6b8208e2ec6b/kube-multus/0.log"
Sep 17 17:09:46.447811 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:46.447781    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-z77cp_f3d4b959-b664-439d-b7a8-f4120b31a45d/network-metrics-daemon/0.log"
Sep 17 17:09:46.465657 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:46.465627    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-z77cp_f3d4b959-b664-439d-b7a8-f4120b31a45d/kube-rbac-proxy/0.log"
Sep 17 17:09:47.201808 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:47.201773    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22fdn_68559fca-75ed-4ee8-bedd-57b04fbd3f4d/ovn-controller/0.log"
Sep 17 17:09:47.218326 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:47.218303    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22fdn_68559fca-75ed-4ee8-bedd-57b04fbd3f4d/ovn-acl-logging/1.log"
Sep 17 17:09:47.218475 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:47.218341    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22fdn_68559fca-75ed-4ee8-bedd-57b04fbd3f4d/ovn-acl-logging/0.log"
Sep 17 17:09:47.235487 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:47.235460    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22fdn_68559fca-75ed-4ee8-bedd-57b04fbd3f4d/kube-rbac-proxy-node/0.log"
Sep 17 17:09:47.255903 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:47.255841    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22fdn_68559fca-75ed-4ee8-bedd-57b04fbd3f4d/kube-rbac-proxy-ovn-metrics/0.log"
Sep 17 17:09:47.273469 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:47.273443    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22fdn_68559fca-75ed-4ee8-bedd-57b04fbd3f4d/northd/0.log"
Sep 17 17:09:47.291774 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:47.291751    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22fdn_68559fca-75ed-4ee8-bedd-57b04fbd3f4d/nbdb/0.log"
Sep 17 17:09:47.310204 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:47.310176    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22fdn_68559fca-75ed-4ee8-bedd-57b04fbd3f4d/sbdb/0.log"
Sep 17 17:09:47.363715 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:47.363683    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-22fdn_68559fca-75ed-4ee8-bedd-57b04fbd3f4d/ovnkube-controller/0.log"
Sep 17 17:09:48.688211 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:48.688174    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-nxc7t_c3e3bba0-3038-411b-8194-5a88b7a0f443/network-check-target-container/0.log"
Sep 17 17:09:49.585127 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:49.585100    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-8flbg_c77bcca8-72d5-49e4-89cc-c80e9492b22f/iptables-alerter/0.log"
Sep 17 17:09:50.257402 ip-10-0-130-106 kubenswrapper[2586]: I0917 17:09:50.257326    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-26cx9_50ca1290-c198-4a93-9246-9707e5df2568/tuned/0.log"