Sep 15 16:24:17.613891 ip-10-0-141-35 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 15 16:24:17.613901 ip-10-0-141-35 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 15 16:24:17.613908 ip-10-0-141-35 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 15 16:24:17.614119 ip-10-0-141-35 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 15 16:24:27.697100 ip-10-0-141-35 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found.
Sep 15 16:24:27.697110 ip-10-0-141-35 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 79a80b1193674eeb9b68cc6daff2e204 --
Sep 15 16:26:23.270065 ip-10-0-141-35 systemd[1]: Starting Kubernetes Kubelet...
Sep 15 16:26:23.684005 ip-10-0-141-35 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 16:26:23.684005 ip-10-0-141-35 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 15 16:26:23.684005 ip-10-0-141-35 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 16:26:23.684005 ip-10-0-141-35 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 15 16:26:23.684005 ip-10-0-141-35 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 16:26:23.684602 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.684010    2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 15 16:26:23.689535 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689515    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 16:26:23.689535 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689533    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 16:26:23.689535 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689538    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 16:26:23.689535 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689542    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 16:26:23.689761 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689547    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 16:26:23.689761 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689551    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 16:26:23.689761 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689555    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 16:26:23.689761 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689559    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 16:26:23.689761 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689562    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 16:26:23.689761 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689566    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 16:26:23.689761 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689570    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 16:26:23.689761 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689574    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 16:26:23.689761 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689577    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 16:26:23.689761 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689601    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 16:26:23.689761 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689607    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 16:26:23.689761 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689611    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 16:26:23.689761 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689616    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 16:26:23.689761 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689620    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 16:26:23.689761 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689625    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 16:26:23.689761 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689629    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 16:26:23.689761 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689633    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 16:26:23.689761 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689637    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 16:26:23.689761 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689641    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 16:26:23.689761 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689645    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 16:26:23.690559 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689649    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 16:26:23.690559 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689653    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 16:26:23.690559 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689659    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 16:26:23.690559 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689665    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 16:26:23.690559 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689671    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 16:26:23.690559 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689675    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 16:26:23.690559 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689680    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 16:26:23.690559 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689685    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 16:26:23.690559 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689690    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 16:26:23.690559 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689694    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 16:26:23.690559 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689698    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 16:26:23.690559 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689702    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 16:26:23.690559 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689706    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 16:26:23.690559 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689710    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 16:26:23.690559 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689715    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 16:26:23.690559 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689719    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 16:26:23.690559 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689723    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 16:26:23.690559 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689727    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 16:26:23.690559 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689731    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 16:26:23.691376 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689735    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 16:26:23.691376 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689739    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 16:26:23.691376 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689743    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 16:26:23.691376 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689747    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 16:26:23.691376 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689752    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 16:26:23.691376 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689756    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 16:26:23.691376 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689760    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 16:26:23.691376 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689763    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 16:26:23.691376 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689768    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 16:26:23.691376 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689772    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 16:26:23.691376 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689777    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 16:26:23.691376 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689803    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 16:26:23.691376 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689808    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 16:26:23.691376 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689812    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 16:26:23.691376 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689816    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 16:26:23.691376 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689820    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 16:26:23.691376 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689824    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 16:26:23.691376 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689829    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 16:26:23.691376 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689833    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 16:26:23.691376 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689837    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 16:26:23.692277 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689841    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 16:26:23.692277 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689845    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 16:26:23.692277 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689849    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 16:26:23.692277 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689852    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 16:26:23.692277 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689857    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 16:26:23.692277 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689861    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 16:26:23.692277 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689865    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 16:26:23.692277 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689869    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 16:26:23.692277 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689873    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 16:26:23.692277 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689879    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 16:26:23.692277 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689883    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 16:26:23.692277 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689887    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 16:26:23.692277 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689891    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 16:26:23.692277 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689895    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 16:26:23.692277 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689899    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 16:26:23.692277 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689903    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 16:26:23.692277 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689907    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 16:26:23.692277 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689911    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 16:26:23.692277 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689915    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 16:26:23.693080 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689919    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 16:26:23.693080 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689927    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 16:26:23.693080 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689933    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 16:26:23.693080 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689937    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 16:26:23.693080 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689941    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 16:26:23.693080 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689946    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 16:26:23.693080 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689950    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 16:26:23.693080 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689955    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 16:26:23.693080 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689959    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 16:26:23.693080 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689963    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 16:26:23.693080 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689969    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 16:26:23.693080 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689973    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 16:26:23.693080 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689978    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 16:26:23.693080 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689982    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 16:26:23.693080 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689987    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 16:26:23.693080 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689991    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 16:26:23.693080 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.689995    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 16:26:23.693080 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691116    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 16:26:23.693080 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691126    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 16:26:23.693911 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691131    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 16:26:23.693911 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691135    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 16:26:23.693911 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691139    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 16:26:23.693911 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691145    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 16:26:23.693911 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691149    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 16:26:23.693911 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691153    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 16:26:23.693911 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691157    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 16:26:23.693911 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691161    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 16:26:23.693911 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691165    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 16:26:23.693911 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691172    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 16:26:23.693911 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691177    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 16:26:23.693911 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691182    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 16:26:23.693911 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691186    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 16:26:23.693911 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691190    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 16:26:23.693911 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691194    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 16:26:23.693911 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691200    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 16:26:23.693911 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691204    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 16:26:23.693911 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691208    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 16:26:23.693911 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691213    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 16:26:23.694440 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691217    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 16:26:23.694440 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691223    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 16:26:23.694440 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691229    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 16:26:23.694440 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691243    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 16:26:23.694440 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691248    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 16:26:23.694440 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691253    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 16:26:23.694440 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691257    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 16:26:23.694440 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691261    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 16:26:23.694440 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691265    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 16:26:23.694440 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691269    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 16:26:23.694440 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691273    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 16:26:23.694440 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691277    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 16:26:23.694440 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691281    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 16:26:23.694440 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691285    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 16:26:23.694440 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691289    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 16:26:23.694440 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691292    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 16:26:23.694440 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691296    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 16:26:23.694440 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691300    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 16:26:23.694440 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691304    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 16:26:23.694440 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691308    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 16:26:23.695005 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691312    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 16:26:23.695005 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691316    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 16:26:23.695005 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691321    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 16:26:23.695005 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691325    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 16:26:23.695005 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691329    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 16:26:23.695005 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691332    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 16:26:23.695005 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691336    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 16:26:23.695005 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691340    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 16:26:23.695005 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691344    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 16:26:23.695005 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691350    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 16:26:23.695005 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691354    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 16:26:23.695005 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691358    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 16:26:23.695005 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691362    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 16:26:23.695005 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691366    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 16:26:23.695005 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691370    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 16:26:23.695005 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691373    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 16:26:23.695005 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691378    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 16:26:23.695005 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691381    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 16:26:23.695005 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691385    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 16:26:23.695478 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691389    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 16:26:23.695478 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691393    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 16:26:23.695478 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691397    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 16:26:23.695478 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691401    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 16:26:23.695478 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691405    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 16:26:23.695478 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691409    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 16:26:23.695478 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691413    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 16:26:23.695478 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691417    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 16:26:23.695478 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691421    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 16:26:23.695478 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691425    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 16:26:23.695478 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691429    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 16:26:23.695478 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691433    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 16:26:23.695478 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691437    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 16:26:23.695478 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691441    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 16:26:23.695478 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691446    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 16:26:23.695478 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691450    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 16:26:23.695478 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691454    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 16:26:23.695478 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691457    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 16:26:23.695478 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691462    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 16:26:23.695478 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691466    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 16:26:23.695991 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691469    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 16:26:23.695991 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691473    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 16:26:23.695991 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691478    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 16:26:23.695991 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691483    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 16:26:23.695991 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691486    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 16:26:23.695991 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691490    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 16:26:23.695991 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691494    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 16:26:23.695991 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691501    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 16:26:23.695991 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691506    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 16:26:23.695991 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691511    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 16:26:23.695991 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691517    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 16:26:23.695991 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691522    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 16:26:23.695991 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691525    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 16:26:23.695991 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691529    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 16:26:23.695991 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691534    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 16:26:23.695991 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691539    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 16:26:23.695991 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691542    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 16:26:23.695991 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691546    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 16:26:23.695991 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.691550    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 16:26:23.695991 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692128    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 16:26:23.696517 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692136    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 16:26:23.696517 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692140    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 16:26:23.696517 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692144    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 16:26:23.696517 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692148    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 16:26:23.696517 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692153    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 16:26:23.696517 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692158    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 16:26:23.696517 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692162    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 16:26:23.696517 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692166    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 16:26:23.696517 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692170    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 16:26:23.696517 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692174    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 16:26:23.696517 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692178    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 16:26:23.696517 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692182    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 16:26:23.696517 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692186    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 16:26:23.696517 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692190    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 16:26:23.696517 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692194    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 16:26:23.696517 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692199    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 16:26:23.696517 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692204    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 16:26:23.696517 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692207    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 16:26:23.696517 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692211    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 16:26:23.696517 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692215    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 16:26:23.697131 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692219    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 16:26:23.697131 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692223    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 16:26:23.697131 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692227    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 16:26:23.697131 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692231    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 16:26:23.697131 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692234    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 16:26:23.697131 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692238    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 16:26:23.697131 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692243    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 16:26:23.697131 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692247    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 16:26:23.697131 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692251    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 16:26:23.697131 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692255    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 16:26:23.697131 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692259    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 16:26:23.697131 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692263    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 16:26:23.697131 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692267    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 16:26:23.697131 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692271    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 16:26:23.697131 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692274    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 16:26:23.697131 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692278    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 16:26:23.697131 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692283    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 16:26:23.697131 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692289    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 16:26:23.697131 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692292    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 16:26:23.697584 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692298    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 16:26:23.697584 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692304    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 16:26:23.697584 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692308    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 16:26:23.697584 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692315    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 16:26:23.697584 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692320    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 16:26:23.697584 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692324    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 16:26:23.697584 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692329    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 16:26:23.697584 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692333    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 16:26:23.697584 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692338    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 16:26:23.697584 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692343    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 16:26:23.697584 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692347    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 16:26:23.697584 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692352    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 16:26:23.697584 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692357    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 16:26:23.697584 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692361    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 16:26:23.697584 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692365    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 16:26:23.697584 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692369    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 16:26:23.697584 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692374    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 16:26:23.697584 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692377    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 16:26:23.697584 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692381    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 16:26:23.698171 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692386    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 16:26:23.698171 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692390    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 16:26:23.698171 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692394    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 16:26:23.698171 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692398    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 16:26:23.698171 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692402    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 16:26:23.698171 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692406    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 16:26:23.698171 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692410    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 16:26:23.698171 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692414    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 16:26:23.698171 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692418    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 16:26:23.698171 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692422    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 16:26:23.698171 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692426    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 16:26:23.698171 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692430    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 16:26:23.698171 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692434    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 16:26:23.698171 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692438    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 16:26:23.698171 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692442    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 16:26:23.698171 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692446    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 16:26:23.698171 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692451    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 16:26:23.698171 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692455    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 16:26:23.698171 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692459    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 16:26:23.698171 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692463    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 16:26:23.698644 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692467    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 16:26:23.698644 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692471    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 16:26:23.698644 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692475    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 16:26:23.698644 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692480    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 16:26:23.698644 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692484    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 16:26:23.698644 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692488    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 16:26:23.698644 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692492    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 16:26:23.698644 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692496    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 16:26:23.698644 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692500    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 16:26:23.698644 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692504    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 16:26:23.698644 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692508    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 16:26:23.698644 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692511    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 16:26:23.698644 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692515    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 16:26:23.698644 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692519    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 16:26:23.698644 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692523    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 16:26:23.698644 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692527    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 16:26:23.698644 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692531    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 16:26:23.698644 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692535    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 16:26:23.698644 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692539    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 16:26:23.698644 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.692543    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 16:26:23.699149 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692642    2579 flags.go:64] FLAG: --address="0.0.0.0"
Sep 15 16:26:23.699149 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692694    2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 15 16:26:23.699149 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692702    2579 flags.go:64] FLAG: --anonymous-auth="true"
Sep 15 16:26:23.699149 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692708    2579 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 15 16:26:23.699149 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692714    2579 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 15 16:26:23.699149 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692719    2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 15 16:26:23.699149 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692725    2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 15 16:26:23.699149 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692731    2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 15 16:26:23.699149 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692735    2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 15 16:26:23.699149 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692740    2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 15 16:26:23.699149 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692745    2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 15 16:26:23.699149 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692750    2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 15 16:26:23.699149 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692756    2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 15 16:26:23.699149 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692760    2579 flags.go:64] FLAG: --cgroup-root=""
Sep 15 16:26:23.699149 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692764    2579 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 15 16:26:23.699149 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692769    2579 flags.go:64] FLAG: --client-ca-file=""
Sep 15 16:26:23.699149 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692775    2579 flags.go:64] FLAG: --cloud-provider="external"
Sep 15 16:26:23.699149 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692798    2579 flags.go:64] FLAG: --cluster-dns="[]"
Sep 15 16:26:23.699149 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692805    2579 flags.go:64] FLAG: --cluster-domain=""
Sep 15 16:26:23.699149 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692809    2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 15 16:26:23.699149 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692814    2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 15 16:26:23.699149 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692819    2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 15 16:26:23.699149 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692824    2579 flags.go:64] FLAG: --container-log-max-files="5"
Sep 15 16:26:23.699149 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692829    2579 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 15 16:26:23.699711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692834    2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 15 16:26:23.699711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692838    2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 15 16:26:23.699711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692846    2579 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 15 16:26:23.699711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692850    2579 flags.go:64] FLAG: --contention-profiling="false"
Sep 15 16:26:23.699711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692855    2579 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 15 16:26:23.699711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692859    2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 15 16:26:23.699711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692864    2579 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 15 16:26:23.699711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692868    2579 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 15 16:26:23.699711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692874    2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 15 16:26:23.699711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692879    2579 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 15 16:26:23.699711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692883    2579 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 15 16:26:23.699711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692887    2579 flags.go:64] FLAG: --enable-load-reader="false"
Sep 15 16:26:23.699711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692891    2579 flags.go:64] FLAG: --enable-server="true"
Sep 15 16:26:23.699711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692896    2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 15 16:26:23.699711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692902    2579 flags.go:64] FLAG: --event-burst="100"
Sep 15 16:26:23.699711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692906    2579 flags.go:64] FLAG: --event-qps="50"
Sep 15 16:26:23.699711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692911    2579 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 15 16:26:23.699711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692915    2579 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 15 16:26:23.699711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692922    2579 flags.go:64] FLAG: --eviction-hard=""
Sep 15 16:26:23.699711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692927    2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 15 16:26:23.699711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692932    2579 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 15 16:26:23.699711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692937    2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 15 16:26:23.699711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692941    2579 flags.go:64] FLAG: --eviction-soft=""
Sep 15 16:26:23.699711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692946    2579 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 15 16:26:23.699711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692950    2579 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 15 16:26:23.700329 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692955    2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 15 16:26:23.700329 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692959    2579 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 15 16:26:23.700329 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692964    2579 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 15 16:26:23.700329 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692968    2579 flags.go:64] FLAG: --fail-swap-on="true"
Sep 15 16:26:23.700329 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692972    2579 flags.go:64] FLAG: --feature-gates=""
Sep 15 16:26:23.700329 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692978    2579 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 15 16:26:23.700329 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692983    2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 15 16:26:23.700329 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692987    2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 15 16:26:23.700329 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692992    2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 15 16:26:23.700329 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.692998    2579 flags.go:64] FLAG: --healthz-port="10248"
Sep 15 16:26:23.700329 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693002    2579 flags.go:64] FLAG: --help="false"
Sep 15 16:26:23.700329 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693007    2579 flags.go:64] FLAG: --hostname-override="ip-10-0-141-35.ec2.internal"
Sep 15 16:26:23.700329 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693012    2579 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 15 16:26:23.700329 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693016    2579 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 15 16:26:23.700329 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693021    2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 15 16:26:23.700329 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693026    2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 15 16:26:23.700329 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693031    2579 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 15 16:26:23.700329 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693035    2579 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 15 16:26:23.700329 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693040    2579 flags.go:64] FLAG: --image-service-endpoint=""
Sep 15 16:26:23.700329 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693044    2579 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 15 16:26:23.700329 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693048    2579 flags.go:64] FLAG: --kube-api-burst="100"
Sep 15 16:26:23.700329 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693053    2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 15 16:26:23.700329 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693058    2579 flags.go:64] FLAG: --kube-api-qps="50"
Sep 15 16:26:23.700329 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693062    2579 flags.go:64] FLAG: --kube-reserved=""
Sep 15 16:26:23.700993 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693067    2579 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 15 16:26:23.700993 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693072    2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 15 16:26:23.700993 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693077    2579 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 15 16:26:23.700993 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693081    2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 15 16:26:23.700993 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693085    2579 flags.go:64] FLAG: --lock-file=""
Sep 15 16:26:23.700993 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693089    2579 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 15 16:26:23.700993 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693093    2579 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 15 16:26:23.700993 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693097    2579 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 15 16:26:23.700993 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693104    2579 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 15 16:26:23.700993 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693110    2579 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 15 16:26:23.700993 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693114    2579 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 15 16:26:23.700993 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693119    2579 flags.go:64] FLAG: --logging-format="text"
Sep 15 16:26:23.700993 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693123    2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 15 16:26:23.700993 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693128    2579 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 15 16:26:23.700993 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693133    2579 flags.go:64] FLAG: --manifest-url=""
Sep 15 16:26:23.700993 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693137    2579 flags.go:64] FLAG: --manifest-url-header=""
Sep 15 16:26:23.700993 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693143    2579 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 15 16:26:23.700993 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693150    2579 flags.go:64] FLAG: --max-open-files="1000000"
Sep 15 16:26:23.700993 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693156    2579 flags.go:64] FLAG: --max-pods="110"
Sep 15 16:26:23.700993 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693160    2579 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 15 16:26:23.700993 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693165    2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 15 16:26:23.700993 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693169    2579 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 15 16:26:23.700993 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693174    2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 15 16:26:23.700993 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693178    2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 15 16:26:23.700993 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693183    2579 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 15 16:26:23.701631 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693187    2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 15 16:26:23.701631 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693198    2579 flags.go:64] FLAG: --node-status-max-images="50"
Sep 15 16:26:23.701631 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693202    2579 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 15 16:26:23.701631 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693207    2579 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 15 16:26:23.701631 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693212    2579 flags.go:64] FLAG: --pod-cidr=""
Sep 15 16:26:23.701631 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693216    2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 15 16:26:23.701631 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693225    2579 flags.go:64] FLAG: --pod-manifest-path=""
Sep 15 16:26:23.701631 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693229    2579 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 15 16:26:23.701631 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693234    2579 flags.go:64] FLAG: --pods-per-core="0"
Sep 15 16:26:23.701631 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693240    2579 flags.go:64] FLAG: --port="10250"
Sep 15 16:26:23.701631 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693245    2579 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 15 16:26:23.701631 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693249    2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-094cfc6d897eb9b2d"
Sep 15 16:26:23.701631 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693254    2579 flags.go:64] FLAG: --qos-reserved=""
Sep 15 16:26:23.701631 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693259    2579 flags.go:64] FLAG: --read-only-port="10255"
Sep 15 16:26:23.701631 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693263    2579 flags.go:64] FLAG: --register-node="true"
Sep 15 16:26:23.701631 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693268    2579 flags.go:64] FLAG: --register-with-taints=""
Sep 15 16:26:23.701631 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693273    2579 flags.go:64] FLAG: --registry-burst="10"
Sep 15 16:26:23.701631 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693278    2579 flags.go:64] FLAG: --registry-qps="5"
Sep 15 16:26:23.701631 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693283    2579 flags.go:64] FLAG: --reserved-cpus=""
Sep 15 16:26:23.701631 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693287    2579 flags.go:64] FLAG: --reserved-memory=""
Sep 15 16:26:23.701631 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693292    2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 15 16:26:23.701631 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693297    2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 15 16:26:23.701631 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693301    2579 flags.go:64] FLAG: --rotate-certificates="false"
Sep 15 16:26:23.701631 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693305    2579 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 15 16:26:23.701631 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693312    2579 flags.go:64] FLAG: --runonce="false"
Sep 15 16:26:23.702253 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693316    2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 15 16:26:23.702253 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693321    2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 15 16:26:23.702253 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693325    2579 flags.go:64] FLAG: --seccomp-default="false"
Sep 15 16:26:23.702253 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693330    2579 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 15 16:26:23.702253 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693334    2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 15 16:26:23.702253 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693338    2579 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 15 16:26:23.702253 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693343    2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 15 16:26:23.702253 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693347    2579 flags.go:64] FLAG: --storage-driver-password="root"
Sep 15 16:26:23.702253 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693352    2579 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 15 16:26:23.702253 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693356    2579 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 15 16:26:23.702253 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693361    2579 flags.go:64] FLAG: --storage-driver-user="root"
Sep 15 16:26:23.702253 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693365    2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 15 16:26:23.702253 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693369    2579 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 15 16:26:23.702253 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693374    2579 flags.go:64] FLAG: --system-cgroups=""
Sep 15 16:26:23.702253 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693378    2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 15 16:26:23.702253 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693386    2579 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 15 16:26:23.702253 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693393    2579 flags.go:64] FLAG: --tls-cert-file=""
Sep 15 16:26:23.702253 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693397    2579 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 15 16:26:23.702253 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693403    2579 flags.go:64] FLAG: --tls-min-version=""
Sep 15 16:26:23.702253 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693407    2579 flags.go:64] FLAG: --tls-private-key-file=""
Sep 15 16:26:23.702253 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693412    2579 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 15 16:26:23.702253 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693416    2579 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 15 16:26:23.702253 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693421    2579 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 15 16:26:23.702253 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693425    2579 flags.go:64] FLAG: --v="2"
Sep 15 16:26:23.702253 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693431    2579 flags.go:64] FLAG: --version="false"
Sep 15 16:26:23.702908 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693438    2579 flags.go:64] FLAG: --vmodule=""
Sep 15 16:26:23.702908 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693443    2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 15 16:26:23.702908 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.693448    2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 15 16:26:23.702908 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693608    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 16:26:23.702908 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693614    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 16:26:23.702908 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693619    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 16:26:23.702908 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693625    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 16:26:23.702908 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693629    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 16:26:23.702908 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693633    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 16:26:23.702908 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693637    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 16:26:23.702908 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693642    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 16:26:23.702908 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693646    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 16:26:23.702908 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693649    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 16:26:23.702908 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693653    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 16:26:23.702908 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693657    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 16:26:23.702908 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693661    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 16:26:23.702908 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693665    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 16:26:23.702908 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693669    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 16:26:23.702908 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693673    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 16:26:23.702908 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693677    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 16:26:23.703381 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693681    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 16:26:23.703381 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693685    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 16:26:23.703381 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693690    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 16:26:23.703381 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693695    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 16:26:23.703381 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693699    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 16:26:23.703381 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693703    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 16:26:23.703381 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693706    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 16:26:23.703381 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693710    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 16:26:23.703381 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693714    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 16:26:23.703381 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693718    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 16:26:23.703381 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693724    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 16:26:23.703381 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693730    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 16:26:23.703381 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693734    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 16:26:23.703381 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693738    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 16:26:23.703381 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693743    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 16:26:23.703381 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693747    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 16:26:23.703381 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693750    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 16:26:23.703381 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693755    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 16:26:23.703381 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693760    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 16:26:23.703850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693764    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 16:26:23.703850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693768    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 16:26:23.703850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693772    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 16:26:23.703850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693776    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 16:26:23.703850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693798    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 16:26:23.703850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693801    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 16:26:23.703850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693805    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 16:26:23.703850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693809    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 16:26:23.703850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693814    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 16:26:23.703850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693817    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 16:26:23.703850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693821    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 16:26:23.703850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693825    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 16:26:23.703850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693829    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 16:26:23.703850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693833    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 16:26:23.703850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693836    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 16:26:23.703850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693840    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 16:26:23.703850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693845    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 16:26:23.703850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693849    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 16:26:23.703850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693854    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 16:26:23.703850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693858    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 16:26:23.704328 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693862    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 16:26:23.704328 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693865    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 16:26:23.704328 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693869    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 16:26:23.704328 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693874    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 16:26:23.704328 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693879    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 16:26:23.704328 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693883    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 16:26:23.704328 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693887    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 16:26:23.704328 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693891    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 16:26:23.704328 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693896    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 16:26:23.704328 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693900    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 16:26:23.704328 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693904    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 16:26:23.704328 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693912    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 16:26:23.704328 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693916    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 16:26:23.704328 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693920    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 16:26:23.704328 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693924    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 16:26:23.704328 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693927    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 16:26:23.704328 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693931    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 16:26:23.704328 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693935    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 16:26:23.704328 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693938    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 16:26:23.704328 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693943    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 16:26:23.704850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693946    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 16:26:23.704850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693950    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 16:26:23.704850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693955    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 16:26:23.704850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693958    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 16:26:23.704850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693963    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 16:26:23.704850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693967    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 16:26:23.704850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693970    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 16:26:23.704850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693974    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 16:26:23.704850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693980    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 16:26:23.704850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693984    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 16:26:23.704850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693988    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 16:26:23.704850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693992    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 16:26:23.704850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.693996    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 16:26:23.704850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.694002    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 16:26:23.704850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.694007    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 16:26:23.704850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.694012    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 16:26:23.704850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.694016    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 16:26:23.704850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.694020    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 16:26:23.704850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.694024    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 16:26:23.704850 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.694028    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 16:26:23.705320 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.694033    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 16:26:23.705320 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.694037    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 16:26:23.705320 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.694043    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 16:26:23.705320 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.694810    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 16:26:23.705320 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.702159    2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 15 16:26:23.705320 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.702173    2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 15 16:26:23.705320 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.703065    2579 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 15 16:26:23.705320 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.703077    2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 15 16:26:23.705320 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703134    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 16:26:23.705320 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703138    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 16:26:23.705320 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703141    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 16:26:23.705320 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703144    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 16:26:23.705320 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703147    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 16:26:23.705320 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703150    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 16:26:23.705657 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703153    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 16:26:23.705657 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703156    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 16:26:23.705657 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703159    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 16:26:23.705657 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703161    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 16:26:23.705657 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703164    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 16:26:23.705657 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703167    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 16:26:23.705657 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703169    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 16:26:23.705657 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703172    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 16:26:23.705657 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703174    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 16:26:23.705657 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703176    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 16:26:23.705657 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703179    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 16:26:23.705657 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703181    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 16:26:23.705657 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703183    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 16:26:23.705657 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703186    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 16:26:23.705657 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703188    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 16:26:23.705657 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703191    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 16:26:23.705657 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703193    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 16:26:23.705657 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703195    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 16:26:23.705657 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703198    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 16:26:23.705657 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703201    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 16:26:23.706144 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703204    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 16:26:23.706144 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703207    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 16:26:23.706144 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703209    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 16:26:23.706144 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703212    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 16:26:23.706144 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703214    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 16:26:23.706144 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703217    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 16:26:23.706144 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703219    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 16:26:23.706144 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703222    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 16:26:23.706144 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703224    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 16:26:23.706144 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703226    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 16:26:23.706144 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703230    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 16:26:23.706144 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703240    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 16:26:23.706144 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703243    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 16:26:23.706144 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703246    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 16:26:23.706144 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703249    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 16:26:23.706144 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703251    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 16:26:23.706144 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703254    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 16:26:23.706144 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703256    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 16:26:23.706144 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703259    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 16:26:23.706144 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703261    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 16:26:23.706615 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703264    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 16:26:23.706615 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703266    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 16:26:23.706615 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703269    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 16:26:23.706615 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703271    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 16:26:23.706615 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703273    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 16:26:23.706615 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703276    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 16:26:23.706615 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703278    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 16:26:23.706615 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703280    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 16:26:23.706615 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703283    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 16:26:23.706615 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703285    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 16:26:23.706615 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703288    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 16:26:23.706615 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703291    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 16:26:23.706615 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703294    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 16:26:23.706615 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703296    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 16:26:23.706615 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703299    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 16:26:23.706615 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703301    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 16:26:23.706615 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703304    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 16:26:23.706615 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703307    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 16:26:23.706615 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703309    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 16:26:23.706615 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703312    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 16:26:23.707125 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703314    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 16:26:23.707125 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703317    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 16:26:23.707125 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703320    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 16:26:23.707125 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703324    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 16:26:23.707125 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703328    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 16:26:23.707125 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703331    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 16:26:23.707125 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703333    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 16:26:23.707125 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703336    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 16:26:23.707125 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703339    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 16:26:23.707125 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703342    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 16:26:23.707125 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703344    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 16:26:23.707125 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703347    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 16:26:23.707125 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703349    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 16:26:23.707125 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703352    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 16:26:23.707125 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703354    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 16:26:23.707125 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703357    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 16:26:23.707125 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703359    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 16:26:23.707125 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703362    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 16:26:23.707125 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703364    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 16:26:23.707575 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703367    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 16:26:23.707575 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703369    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 16:26:23.707575 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703372    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 16:26:23.707575 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703374    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 16:26:23.707575 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703377    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 16:26:23.707575 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703379    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 16:26:23.707575 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703382    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 16:26:23.707575 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703385    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 16:26:23.707575 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703387    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 16:26:23.707575 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703390    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 16:26:23.707575 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703393    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 16:26:23.707575 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703395    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 16:26:23.707575 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703398    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 16:26:23.707575 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703400    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 16:26:23.707575 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.703405    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 16:26:23.707947 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703534    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 16:26:23.707947 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703539    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 16:26:23.707947 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703541    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 16:26:23.707947 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703544    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 16:26:23.707947 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703546    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 16:26:23.707947 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703549    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 16:26:23.707947 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703552    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 16:26:23.707947 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703554    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 16:26:23.707947 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703556    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 16:26:23.707947 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703559    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 16:26:23.707947 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703561    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 16:26:23.707947 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703563    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 16:26:23.707947 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703566    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 16:26:23.707947 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703568    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 16:26:23.707947 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703570    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 16:26:23.707947 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703573    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 16:26:23.707947 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703575    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 16:26:23.707947 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703578    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 16:26:23.707947 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703580    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 16:26:23.708401 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703582    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 16:26:23.708401 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703585    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 16:26:23.708401 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703587    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 16:26:23.708401 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703590    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 16:26:23.708401 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703592    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 16:26:23.708401 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703595    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 16:26:23.708401 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703597    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 16:26:23.708401 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703600    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 16:26:23.708401 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703602    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 16:26:23.708401 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703605    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 16:26:23.708401 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703607    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 16:26:23.708401 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703610    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 16:26:23.708401 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703612    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 16:26:23.708401 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703615    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 16:26:23.708401 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703617    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 16:26:23.708401 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703620    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 16:26:23.708401 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703622    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 16:26:23.708401 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703624    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 16:26:23.708401 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703627    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 16:26:23.708401 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703629    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 16:26:23.708889 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703632    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 16:26:23.708889 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703634    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 16:26:23.708889 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703637    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 16:26:23.708889 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703639    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 16:26:23.708889 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703641    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 16:26:23.708889 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703644    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 16:26:23.708889 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703646    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 16:26:23.708889 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703648    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 16:26:23.708889 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703651    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 16:26:23.708889 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703653    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 16:26:23.708889 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703655    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 16:26:23.708889 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703658    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 16:26:23.708889 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703661    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 16:26:23.708889 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703663    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 16:26:23.708889 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703666    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 16:26:23.708889 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703670    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 16:26:23.708889 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703672    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 16:26:23.708889 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703675    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 16:26:23.708889 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703679    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 16:26:23.709335 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703682    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 16:26:23.709335 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703685    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 16:26:23.709335 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703688    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 16:26:23.709335 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703697    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 16:26:23.709335 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703699    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 16:26:23.709335 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703701    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 16:26:23.709335 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703704    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 16:26:23.709335 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703706    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 16:26:23.709335 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703708    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 16:26:23.709335 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703711    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 16:26:23.709335 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703713    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 16:26:23.709335 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703716    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 16:26:23.709335 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703718    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 16:26:23.709335 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703721    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 16:26:23.709335 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703724    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 16:26:23.709335 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703726    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 16:26:23.709335 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703728    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 16:26:23.709335 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703731    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 16:26:23.709335 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703733    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 16:26:23.709335 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703736    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 16:26:23.709795 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703738    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 16:26:23.709795 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703741    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 16:26:23.709795 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703743    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 16:26:23.709795 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703746    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 16:26:23.709795 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703748    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 16:26:23.709795 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703750    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 16:26:23.709795 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703753    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 16:26:23.709795 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703755    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 16:26:23.709795 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703757    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 16:26:23.709795 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703760    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 16:26:23.709795 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703762    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 16:26:23.709795 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703764    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 16:26:23.709795 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703767    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 16:26:23.709795 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703769    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 16:26:23.709795 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703772    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 16:26:23.709795 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703774    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 16:26:23.709795 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703800    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 16:26:23.709795 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703804    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 16:26:23.709795 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703808    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 16:26:23.710270 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703811    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 16:26:23.710270 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:23.703813    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 16:26:23.710270 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.703818    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 16:26:23.710270 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.703930    2579 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 15 16:26:23.710270 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.707274    2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 15 16:26:23.710270 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.708419    2579 server.go:1019] "Starting client certificate rotation"
Sep 15 16:26:23.710270 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.708526    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 15 16:26:23.710270 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.708559    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 15 16:26:23.711414 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.711398    2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 15 16:26:23.718119 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.718073    2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 15 16:26:23.729392 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.729377    2579 log.go:25] "Validated CRI v1 runtime API"
Sep 15 16:26:23.734526 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.734512    2579 log.go:25] "Validated CRI v1 image API"
Sep 15 16:26:23.735324 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.735309    2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 15 16:26:23.739512 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.739495    2579 fs.go:135] Filesystem UUIDs: map[3804e7fa-c827-44c1-a165-595a5b94f8a5:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a0b2d0fe-a895-41a5-b081-4ed530c1a6de:/dev/nvme0n1p4]
Sep 15 16:26:23.739579 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.739511    2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 15 16:26:23.744709 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.744609    2579 manager.go:216] Machine: {Timestamp:2026-09-15 16:26:23.74313379 +0000 UTC m=+0.372788133 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3204925 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23018bfcead6cb535d4440290be513 SystemUUID:ec23018b-fcea-d6cb-535d-4440290be513 BootID:79a80b11-9367-4eeb-9b68-cc6daff2e204 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:e1:db:94:1d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:e1:db:94:1d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e6:b5:fe:00:76:21 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 15 16:26:23.744709 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.744706    2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 15 16:26:23.744817 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.744768    2579 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 15 16:26:23.745476 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.745462    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 15 16:26:23.748585 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.748563    2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 15 16:26:23.748711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.748588    2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-35.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 15 16:26:23.750128 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.750118    2579 topology_manager.go:138] "Creating topology manager with none policy"
Sep 15 16:26:23.750163 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.750131    2579 container_manager_linux.go:309] "Creating device plugin manager"
Sep 15 16:26:23.750163 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.750138    2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 15 16:26:23.750979 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.750969    2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 15 16:26:23.752976 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.752966    2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 15 16:26:23.757698 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.757687    2579 state_mem.go:36] "Initialized new in-memory state store"
Sep 15 16:26:23.757819 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.757810    2579 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 15 16:26:23.757853 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.757840    2579 kubelet.go:486] "Attempting to sync node with API server"
Sep 15 16:26:23.757853 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.757850    2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 15 16:26:23.757908 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.757862    2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 15 16:26:23.757908 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.757872    2579 kubelet.go:398] "Adding apiserver pod source"
Sep 15 16:26:23.757908 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.757880    2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 15 16:26:23.758292 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.758279    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 15 16:26:23.758334 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.758306    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 15 16:26:23.760606 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.760584    2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 15 16:26:23.765383 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.765364    2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 15 16:26:23.765466 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.765398    2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 15 16:26:23.767504 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.767487    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 15 16:26:23.767564 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.767510    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 15 16:26:23.767564 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.767520    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 15 16:26:23.767564 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.767528    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 15 16:26:23.767564 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.767540    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 15 16:26:23.767564 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.767548    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 15 16:26:23.767723 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.767567    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 15 16:26:23.767723 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.767574    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 15 16:26:23.767723 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.767583    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 15 16:26:23.767723 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.767591    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 15 16:26:23.767723 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.767604    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 15 16:26:23.767723 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.767612    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 15 16:26:23.767894 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.767763    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-llfvc"
Sep 15 16:26:23.768109 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:23.768092    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 15 16:26:23.768255 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:23.768241    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-35.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 15 16:26:23.769463 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.769451    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 15 16:26:23.769500 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.769465    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 15 16:26:23.773123 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.773107    2579 server.go:1268] "Started kubelet"
Sep 15 16:26:23.773205 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.773154    2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 15 16:26:23.773282 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.773238    2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 15 16:26:23.773337 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.773306    2579 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 15 16:26:23.773801 ip-10-0-141-35 systemd[1]: Started Kubernetes Kubelet.
Sep 15 16:26:23.773920 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.773884    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-llfvc"
Sep 15 16:26:23.774872 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.774808    2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 15 16:26:23.775433 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.775408    2579 server.go:310] "Adding debug handlers to kubelet server"
Sep 15 16:26:23.779217 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.779189    2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-35.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 15 16:26:23.779582 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:23.779561    2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 15 16:26:23.780320 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:23.779224    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-35.ec2.internal.18d58b4b9a0c9949  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-35.ec2.internal,UID:ip-10-0-141-35.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-35.ec2.internal,},FirstTimestamp:2026-09-15 16:26:23.772965193 +0000 UTC m=+0.402619536,LastTimestamp:2026-09-15 16:26:23.772965193 +0000 UTC m=+0.402619536,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-35.ec2.internal,}"
Sep 15 16:26:23.782519 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.782501    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 15 16:26:23.783228 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.783214    2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 15 16:26:23.783342 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:23.783317    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-35.ec2.internal\" not found"
Sep 15 16:26:23.783987 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.783970    2579 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 15 16:26:23.784064 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.783969    2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 15 16:26:23.784064 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.783990    2579 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 15 16:26:23.784159 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.784145    2579 reconstruct.go:97] "Volume reconstruction finished"
Sep 15 16:26:23.784208 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.784160    2579 reconciler.go:29] "Reconciler: start to sync state"
Sep 15 16:26:23.784208 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.784172    2579 factory.go:55] Registering systemd factory
Sep 15 16:26:23.784208 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.784202    2579 factory.go:223] Registration of the systemd container factory successfully
Sep 15 16:26:23.784436 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.784418    2579 factory.go:153] Registering CRI-O factory
Sep 15 16:26:23.784436 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.784438    2579 factory.go:223] Registration of the crio container factory successfully
Sep 15 16:26:23.784581 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.784499    2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 15 16:26:23.784581 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.784515    2579 factory.go:103] Registering Raw factory
Sep 15 16:26:23.784581 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.784528    2579 manager.go:1196] Started watching for new ooms in manager
Sep 15 16:26:23.785591 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.785299    2579 manager.go:349] Starting recovery of all containers
Sep 15 16:26:23.787393 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.787374    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 16:26:23.790929 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:23.790909    2579 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-141-35.ec2.internal\" not found" node="ip-10-0-141-35.ec2.internal"
Sep 15 16:26:23.798694 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.798679    2579 manager.go:354] Recovery completed
Sep 15 16:26:23.802464 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.802453    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 16:26:23.804260 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.804248    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-35.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 16:26:23.804305 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.804273    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-35.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 16:26:23.804305 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.804285    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-35.ec2.internal" event="NodeHasSufficientPID"
Sep 15 16:26:23.804729 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.804717    2579 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 15 16:26:23.804764 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.804730    2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 15 16:26:23.804764 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.804733    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 16:26:23.804764 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.804751    2579 state_mem.go:36] "Initialized new in-memory state store"
Sep 15 16:26:23.805392 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.805377    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-35.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 16:26:23.805457 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.805398    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-35.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 16:26:23.805457 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.805407    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-35.ec2.internal" event="NodeHasSufficientPID"
Sep 15 16:26:23.806883 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.806871    2579 policy_none.go:49] "None policy: Start"
Sep 15 16:26:23.806917 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.806886    2579 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 15 16:26:23.806917 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.806895    2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 15 16:26:23.808820 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.808810    2579 policy_none.go:47] "Start"
Sep 15 16:26:23.847335 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.847322    2579 manager.go:337] "Starting Device Plugin manager"
Sep 15 16:26:23.866219 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:23.847359    2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 15 16:26:23.866219 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.847372    2579 server.go:87] "Starting device plugin registration server"
Sep 15 16:26:23.866219 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.847582    2579 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 15 16:26:23.866219 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.847592    2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 15 16:26:23.866219 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.847710    2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 15 16:26:23.866219 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.847773    2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 15 16:26:23.866219 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.847799    2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 15 16:26:23.866219 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:23.848213    2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 15 16:26:23.866219 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:23.848268    2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-35.ec2.internal\" not found"
Sep 15 16:26:23.901585 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.901566    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 15 16:26:23.902834 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.902813    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 15 16:26:23.902898 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.902865    2579 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 15 16:26:23.902898 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.902887    2579 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 15 16:26:23.902994 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:23.902929    2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 15 16:26:23.905180 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.905165    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 16:26:23.948303 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.948174    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 16:26:23.949006 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.948993    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-35.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 16:26:23.949056 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.949018    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-35.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 16:26:23.949056 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.949028    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-35.ec2.internal" event="NodeHasSufficientPID"
Sep 15 16:26:23.949056 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.949050    2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-35.ec2.internal"
Sep 15 16:26:23.956339 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:23.956326    2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-35.ec2.internal"
Sep 15 16:26:23.956382 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:23.956345    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-35.ec2.internal\": node \"ip-10-0-141-35.ec2.internal\" not found"
Sep 15 16:26:23.971975 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:23.971957    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-35.ec2.internal\" not found"
Sep 15 16:26:24.003630 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.003605    2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-35.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-35.ec2.internal"]
Sep 15 16:26:24.003682 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.003665    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 16:26:24.004872 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.004859    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-35.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 16:26:24.004935 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.004884    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-35.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 16:26:24.004935 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.004899    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-35.ec2.internal" event="NodeHasSufficientPID"
Sep 15 16:26:24.007095 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.007084    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 16:26:24.007253 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.007241    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-35.ec2.internal"
Sep 15 16:26:24.007298 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.007265    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 16:26:24.007855 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.007842    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-35.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 16:26:24.007928 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.007868    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-35.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 16:26:24.007928 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.007881    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-35.ec2.internal" event="NodeHasSufficientPID"
Sep 15 16:26:24.007928 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.007908    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-35.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 16:26:24.008063 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.007951    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-35.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 16:26:24.008063 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.007968    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-35.ec2.internal" event="NodeHasSufficientPID"
Sep 15 16:26:24.010264 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.010248    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-35.ec2.internal"
Sep 15 16:26:24.010348 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.010275    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 16:26:24.010940 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.010913    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-35.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 16:26:24.011037 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.010958    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-35.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 16:26:24.011037 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.010987    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-35.ec2.internal" event="NodeHasSufficientPID"
Sep 15 16:26:24.023273 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:24.023258    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-35.ec2.internal\" not found" node="ip-10-0-141-35.ec2.internal"
Sep 15 16:26:24.027439 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:24.027427    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-35.ec2.internal\" not found" node="ip-10-0-141-35.ec2.internal"
Sep 15 16:26:24.072417 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:24.072394    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-35.ec2.internal\" not found"
Sep 15 16:26:24.085535 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.085520    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3680caa58091ee9b4ffe5030f60f634-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-35.ec2.internal\" (UID: \"c3680caa58091ee9b4ffe5030f60f634\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-35.ec2.internal"
Sep 15 16:26:24.085618 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.085540    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/269bca4735829fdfafb55c4d16c2e698-config\") pod \"kube-apiserver-proxy-ip-10-0-141-35.ec2.internal\" (UID: \"269bca4735829fdfafb55c4d16c2e698\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-35.ec2.internal"
Sep 15 16:26:24.085618 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.085555    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c3680caa58091ee9b4ffe5030f60f634-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-35.ec2.internal\" (UID: \"c3680caa58091ee9b4ffe5030f60f634\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-35.ec2.internal"
Sep 15 16:26:24.172819 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:24.172797    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-35.ec2.internal\" not found"
Sep 15 16:26:24.186120 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.186103    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c3680caa58091ee9b4ffe5030f60f634-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-35.ec2.internal\" (UID: \"c3680caa58091ee9b4ffe5030f60f634\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-35.ec2.internal"
Sep 15 16:26:24.186180 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.186124    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3680caa58091ee9b4ffe5030f60f634-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-35.ec2.internal\" (UID: \"c3680caa58091ee9b4ffe5030f60f634\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-35.ec2.internal"
Sep 15 16:26:24.186180 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.186138    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/269bca4735829fdfafb55c4d16c2e698-config\") pod \"kube-apiserver-proxy-ip-10-0-141-35.ec2.internal\" (UID: \"269bca4735829fdfafb55c4d16c2e698\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-35.ec2.internal"
Sep 15 16:26:24.186243 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.186185    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/269bca4735829fdfafb55c4d16c2e698-config\") pod \"kube-apiserver-proxy-ip-10-0-141-35.ec2.internal\" (UID: \"269bca4735829fdfafb55c4d16c2e698\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-35.ec2.internal"
Sep 15 16:26:24.186243 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.186205    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c3680caa58091ee9b4ffe5030f60f634-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-35.ec2.internal\" (UID: \"c3680caa58091ee9b4ffe5030f60f634\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-35.ec2.internal"
Sep 15 16:26:24.186243 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.186202    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3680caa58091ee9b4ffe5030f60f634-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-35.ec2.internal\" (UID: \"c3680caa58091ee9b4ffe5030f60f634\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-35.ec2.internal"
Sep 15 16:26:24.273508 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:24.273490    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-35.ec2.internal\" not found"
Sep 15 16:26:24.324982 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.324960    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-35.ec2.internal"
Sep 15 16:26:24.330559 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.330545    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-35.ec2.internal"
Sep 15 16:26:24.373869 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:24.373847    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-35.ec2.internal\" not found"
Sep 15 16:26:24.474374 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:24.474358    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-35.ec2.internal\" not found"
Sep 15 16:26:24.574916 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:24.574875    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-35.ec2.internal\" not found"
Sep 15 16:26:24.648105 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.648088    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 16:26:24.683821 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.683805    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-35.ec2.internal"
Sep 15 16:26:24.693834 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.693814    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 15 16:26:24.695163 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.695148    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-35.ec2.internal"
Sep 15 16:26:24.703046 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.703033    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 15 16:26:24.708177 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.708159    2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 15 16:26:24.708263 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.708248    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 15 16:26:24.708324 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.708307    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 15 16:26:24.708353 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.708303    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 15 16:26:24.758469 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.758449    2579 apiserver.go:52] "Watching apiserver"
Sep 15 16:26:24.773415 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.773399    2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 15 16:26:24.773769 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.773753    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-35.ec2.internal","openshift-multus/multus-dqm7s","openshift-multus/network-metrics-daemon-txzqk","openshift-network-operator/iptables-alerter-cs7tq","openshift-ovn-kubernetes/ovnkube-node-b4rdn","kube-system/konnectivity-agent-s7d5t","kube-system/kube-apiserver-proxy-ip-10-0-141-35.ec2.internal","openshift-cluster-node-tuning-operator/tuned-zr9w5","openshift-image-registry/node-ca-8zk2f","openshift-multus/multus-additional-cni-plugins-cvgk4","openshift-network-diagnostics/network-check-target-sqdn2","kube-system/global-pull-secret-syncer-9hbcw","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rw6mp","openshift-dns/node-resolver-btdbz"]
Sep 15 16:26:24.776394 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.776379    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cvgk4"
Sep 15 16:26:24.778600 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.778584    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rw6mp"
Sep 15 16:26:24.778705 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.778688    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 15 16:26:24.778754 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.778738    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bh879\""
Sep 15 16:26:24.778821 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.778688    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 15 16:26:24.779109 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.779084    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 15 16:26:24.779109 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.779102    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 15 16:26:24.779222 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.779162    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 15 16:26:24.780452 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.780410    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-14 16:21:23 +0000 UTC" deadline="2028-06-20 18:38:19.810824974 +0000 UTC"
Sep 15 16:26:24.780525 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.780460    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15458h11m55.030367274s"
Sep 15 16:26:24.780697 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.780678    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 15 16:26:24.780804 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.780774    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:26:24.780920 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:24.780897    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzqk" podUID="1d768ac0-1487-4850-bd9b-c66f73533e97"
Sep 15 16:26:24.781610 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.781217    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 15 16:26:24.781610 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.781236    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 15 16:26:24.781610 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.781391    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-prlxs\""
Sep 15 16:26:24.782763 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.782746    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 15 16:26:24.783047 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.783032    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cs7tq"
Sep 15 16:26:24.785172 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.785152    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jpgw4\""
Sep 15 16:26:24.785387 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.785368    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 15 16:26:24.785387 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.785381    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 15 16:26:24.785523 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.785434    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 15 16:26:24.785523 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.785443    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.787592 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.787572    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 15 16:26:24.787754 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.787733    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 15 16:26:24.787981 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.787966    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 15 16:26:24.788139 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.788127    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 15 16:26:24.788233 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.788217    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 15 16:26:24.788299 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.787975    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mbvxs\""
Sep 15 16:26:24.788630 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.788611    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 15 16:26:24.789147 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.788892    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-s7d5t"
Sep 15 16:26:24.789255 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.789240    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wwqd\" (UniqueName: \"kubernetes.io/projected/1d768ac0-1487-4850-bd9b-c66f73533e97-kube-api-access-6wwqd\") pod \"network-metrics-daemon-txzqk\" (UID: \"1d768ac0-1487-4850-bd9b-c66f73533e97\") " pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:26:24.789318 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.789279    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-host-slash\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.789376 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.789306    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-var-lib-openvswitch\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.789502 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.789482    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-node-log\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.789980 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.789573    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-log-socket\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.789980 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.789619    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.789980 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.789648    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/44ca88f0-c149-443e-a2d1-a30964775818-env-overrides\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.789980 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.789706    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3c332c22-48cd-4420-9a35-e40f98ec4344-device-dir\") pod \"aws-ebs-csi-driver-node-rw6mp\" (UID: \"3c332c22-48cd-4420-9a35-e40f98ec4344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rw6mp"
Sep 15 16:26:24.789980 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.789825    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bf4e438a-fbc7-401f-b97c-bbb7bc4a0495-iptables-alerter-script\") pod \"iptables-alerter-cs7tq\" (UID: \"bf4e438a-fbc7-401f-b97c-bbb7bc4a0495\") " pod="openshift-network-operator/iptables-alerter-cs7tq"
Sep 15 16:26:24.789980 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.789922    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-host-run-netns\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.789980 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.789968    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-host-cni-netd\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.790331 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.789998    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3c332c22-48cd-4420-9a35-e40f98ec4344-socket-dir\") pod \"aws-ebs-csi-driver-node-rw6mp\" (UID: \"3c332c22-48cd-4420-9a35-e40f98ec4344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rw6mp"
Sep 15 16:26:24.790331 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.790053    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-systemd-units\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.790331 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.790084    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-run-systemd\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.790331 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.790117    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sbz4\" (UniqueName: \"kubernetes.io/projected/44ca88f0-c149-443e-a2d1-a30964775818-kube-api-access-2sbz4\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.790331 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.790170    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b17d9664-4ce9-449a-8826-af717fd47f97-system-cni-dir\") pod \"multus-additional-cni-plugins-cvgk4\" (UID: \"b17d9664-4ce9-449a-8826-af717fd47f97\") " pod="openshift-multus/multus-additional-cni-plugins-cvgk4"
Sep 15 16:26:24.790331 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.790227    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b17d9664-4ce9-449a-8826-af717fd47f97-cnibin\") pod \"multus-additional-cni-plugins-cvgk4\" (UID: \"b17d9664-4ce9-449a-8826-af717fd47f97\") " pod="openshift-multus/multus-additional-cni-plugins-cvgk4"
Sep 15 16:26:24.790331 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.790280    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b17d9664-4ce9-449a-8826-af717fd47f97-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cvgk4\" (UID: \"b17d9664-4ce9-449a-8826-af717fd47f97\") " pod="openshift-multus/multus-additional-cni-plugins-cvgk4"
Sep 15 16:26:24.790331 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.790328    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mflr\" (UniqueName: \"kubernetes.io/projected/bf4e438a-fbc7-401f-b97c-bbb7bc4a0495-kube-api-access-9mflr\") pod \"iptables-alerter-cs7tq\" (UID: \"bf4e438a-fbc7-401f-b97c-bbb7bc4a0495\") " pod="openshift-network-operator/iptables-alerter-cs7tq"
Sep 15 16:26:24.790715 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.790355    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-run-openvswitch\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.790715 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.790391    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/44ca88f0-c149-443e-a2d1-a30964775818-ovn-node-metrics-cert\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.790715 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.790417    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-host-kubelet\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.790715 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.790467    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-etc-openvswitch\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.790715 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.790506    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-run-ovn\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.791019 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.790762    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8zk2f"
Sep 15 16:26:24.791120 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.791096    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-host-run-ovn-kubernetes\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.791182 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.791151    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/44ca88f0-c149-443e-a2d1-a30964775818-ovnkube-config\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.791239 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.791181    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qc2kp\" (UniqueName: \"kubernetes.io/projected/b17d9664-4ce9-449a-8826-af717fd47f97-kube-api-access-qc2kp\") pod \"multus-additional-cni-plugins-cvgk4\" (UID: \"b17d9664-4ce9-449a-8826-af717fd47f97\") " pod="openshift-multus/multus-additional-cni-plugins-cvgk4"
Sep 15 16:26:24.791239 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.791218    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b17d9664-4ce9-449a-8826-af717fd47f97-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cvgk4\" (UID: \"b17d9664-4ce9-449a-8826-af717fd47f97\") " pod="openshift-multus/multus-additional-cni-plugins-cvgk4"
Sep 15 16:26:24.791337 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.791261    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b17d9664-4ce9-449a-8826-af717fd47f97-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cvgk4\" (UID: \"b17d9664-4ce9-449a-8826-af717fd47f97\") " pod="openshift-multus/multus-additional-cni-plugins-cvgk4"
Sep 15 16:26:24.791337 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.791284    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3c332c22-48cd-4420-9a35-e40f98ec4344-sys-fs\") pod \"aws-ebs-csi-driver-node-rw6mp\" (UID: \"3c332c22-48cd-4420-9a35-e40f98ec4344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rw6mp"
Sep 15 16:26:24.791337 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.791303    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bf4e438a-fbc7-401f-b97c-bbb7bc4a0495-host-slash\") pod \"iptables-alerter-cs7tq\" (UID: \"bf4e438a-fbc7-401f-b97c-bbb7bc4a0495\") " pod="openshift-network-operator/iptables-alerter-cs7tq"
Sep 15 16:26:24.791337 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.791324    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/44ca88f0-c149-443e-a2d1-a30964775818-ovnkube-script-lib\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.791514 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.791346    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b17d9664-4ce9-449a-8826-af717fd47f97-cni-binary-copy\") pod \"multus-additional-cni-plugins-cvgk4\" (UID: \"b17d9664-4ce9-449a-8826-af717fd47f97\") " pod="openshift-multus/multus-additional-cni-plugins-cvgk4"
Sep 15 16:26:24.791514 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.791368    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3c332c22-48cd-4420-9a35-e40f98ec4344-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rw6mp\" (UID: \"3c332c22-48cd-4420-9a35-e40f98ec4344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rw6mp"
Sep 15 16:26:24.791514 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.791388    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3c332c22-48cd-4420-9a35-e40f98ec4344-etc-selinux\") pod \"aws-ebs-csi-driver-node-rw6mp\" (UID: \"3c332c22-48cd-4420-9a35-e40f98ec4344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rw6mp"
Sep 15 16:26:24.791514 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.791415    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgj9r\" (UniqueName: \"kubernetes.io/projected/3c332c22-48cd-4420-9a35-e40f98ec4344-kube-api-access-wgj9r\") pod \"aws-ebs-csi-driver-node-rw6mp\" (UID: \"3c332c22-48cd-4420-9a35-e40f98ec4344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rw6mp"
Sep 15 16:26:24.791514 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.791435    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d768ac0-1487-4850-bd9b-c66f73533e97-metrics-certs\") pod \"network-metrics-daemon-txzqk\" (UID: \"1d768ac0-1487-4850-bd9b-c66f73533e97\") " pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:26:24.791514 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.791456    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-host-cni-bin\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.791514 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.791481    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3c332c22-48cd-4420-9a35-e40f98ec4344-registration-dir\") pod \"aws-ebs-csi-driver-node-rw6mp\" (UID: \"3c332c22-48cd-4420-9a35-e40f98ec4344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rw6mp"
Sep 15 16:26:24.791837 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.791595    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 15 16:26:24.791837 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.791623    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 15 16:26:24.791837 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.791657    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-tbn2k\""
Sep 15 16:26:24.793567 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.793549    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-btdbz"
Sep 15 16:26:24.794151 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.794135    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 15 16:26:24.794448 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.794429    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tlx92\""
Sep 15 16:26:24.794768 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.794750    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 15 16:26:24.794860 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.794758    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 15 16:26:24.795541 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.795529    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 15 16:26:24.795860 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.795847    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.795996 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.795978    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 15 16:26:24.796108 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.796091    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sqwsx\""
Sep 15 16:26:24.798241 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.798226    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.798294 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.798238    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 15 16:26:24.798294 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.798270    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-nqr2z\""
Sep 15 16:26:24.798588 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.798572    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 15 16:26:24.800382 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.800364    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-v8g46\""
Sep 15 16:26:24.800453 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.800390    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 15 16:26:24.800504 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.800468    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:26:24.800554 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.800512    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 15 16:26:24.800554 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:24.800520    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sqdn2" podUID="ad430be2-7c5f-4402-a5a7-c8b4359b8c02"
Sep 15 16:26:24.802744 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.802726    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:24.802850 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:24.802801    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hbcw" podUID="d1e544d2-21e5-4abd-82d9-33a82470f13c"
Sep 15 16:26:24.839795 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.839767    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7b97s"
Sep 15 16:26:24.847376 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.847360    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7b97s"
Sep 15 16:26:24.884690 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.884672    2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 15 16:26:24.891826 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.891810    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b17d9664-4ce9-449a-8826-af717fd47f97-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cvgk4\" (UID: \"b17d9664-4ce9-449a-8826-af717fd47f97\") " pod="openshift-multus/multus-additional-cni-plugins-cvgk4"
Sep 15 16:26:24.891884 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.891837    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3c332c22-48cd-4420-9a35-e40f98ec4344-sys-fs\") pod \"aws-ebs-csi-driver-node-rw6mp\" (UID: \"3c332c22-48cd-4420-9a35-e40f98ec4344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rw6mp"
Sep 15 16:26:24.891884 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.891856    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-multus-conf-dir\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.891884 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.891872    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3c332c22-48cd-4420-9a35-e40f98ec4344-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rw6mp\" (UID: \"3c332c22-48cd-4420-9a35-e40f98ec4344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rw6mp"
Sep 15 16:26:24.892019 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.891895    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgj9r\" (UniqueName: \"kubernetes.io/projected/3c332c22-48cd-4420-9a35-e40f98ec4344-kube-api-access-wgj9r\") pod \"aws-ebs-csi-driver-node-rw6mp\" (UID: \"3c332c22-48cd-4420-9a35-e40f98ec4344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rw6mp"
Sep 15 16:26:24.892019 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.891918    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdb8314d-7223-4926-b551-8bb1701052fd-tmp-dir\") pod \"node-resolver-btdbz\" (UID: \"fdb8314d-7223-4926-b551-8bb1701052fd\") " pod="openshift-dns/node-resolver-btdbz"
Sep 15 16:26:24.892019 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.891938    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3c332c22-48cd-4420-9a35-e40f98ec4344-sys-fs\") pod \"aws-ebs-csi-driver-node-rw6mp\" (UID: \"3c332c22-48cd-4420-9a35-e40f98ec4344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rw6mp"
Sep 15 16:26:24.892019 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.891950    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-multus-daemon-config\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.892019 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.891972    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-host-run-multus-certs\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.892019 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.891987    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-etc-kubernetes\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.892019 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.891993    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3c332c22-48cd-4420-9a35-e40f98ec4344-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rw6mp\" (UID: \"3c332c22-48cd-4420-9a35-e40f98ec4344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rw6mp"
Sep 15 16:26:24.892277 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892000    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-host\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.892277 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892049    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-host-cni-bin\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.892277 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892070    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wwqd\" (UniqueName: \"kubernetes.io/projected/1d768ac0-1487-4850-bd9b-c66f73533e97-kube-api-access-6wwqd\") pod \"network-metrics-daemon-txzqk\" (UID: \"1d768ac0-1487-4850-bd9b-c66f73533e97\") " pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:26:24.892277 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892088    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-host-var-lib-cni-multus\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.892277 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892114    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-node-log\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.892277 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892131    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-host-cni-bin\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.892277 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892201    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/44ca88f0-c149-443e-a2d1-a30964775818-env-overrides\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.892277 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892221    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3c332c22-48cd-4420-9a35-e40f98ec4344-device-dir\") pod \"aws-ebs-csi-driver-node-rw6mp\" (UID: \"3c332c22-48cd-4420-9a35-e40f98ec4344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rw6mp"
Sep 15 16:26:24.892277 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892221    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-node-log\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.892277 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892237    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv6bf\" (UniqueName: \"kubernetes.io/projected/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-kube-api-access-sv6bf\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.892277 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892258    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-sys\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.892277 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892267    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b17d9664-4ce9-449a-8826-af717fd47f97-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cvgk4\" (UID: \"b17d9664-4ce9-449a-8826-af717fd47f97\") " pod="openshift-multus/multus-additional-cni-plugins-cvgk4"
Sep 15 16:26:24.892277 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892282    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7a83b81c-a949-41e4-9e47-e7305b24d4a7-etc-tuned\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.892873 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892289    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3c332c22-48cd-4420-9a35-e40f98ec4344-device-dir\") pod \"aws-ebs-csi-driver-node-rw6mp\" (UID: \"3c332c22-48cd-4420-9a35-e40f98ec4344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rw6mp"
Sep 15 16:26:24.892873 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892318    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bf4e438a-fbc7-401f-b97c-bbb7bc4a0495-iptables-alerter-script\") pod \"iptables-alerter-cs7tq\" (UID: \"bf4e438a-fbc7-401f-b97c-bbb7bc4a0495\") " pod="openshift-network-operator/iptables-alerter-cs7tq"
Sep 15 16:26:24.892873 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892361    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-host-run-netns\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.892873 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892384    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-host-cni-netd\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.892873 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892397    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-host-run-netns\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.892873 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892399    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3c332c22-48cd-4420-9a35-e40f98ec4344-socket-dir\") pod \"aws-ebs-csi-driver-node-rw6mp\" (UID: \"3c332c22-48cd-4420-9a35-e40f98ec4344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rw6mp"
Sep 15 16:26:24.892873 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892431    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-host-cni-netd\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.892873 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892446    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fdb8314d-7223-4926-b551-8bb1701052fd-hosts-file\") pod \"node-resolver-btdbz\" (UID: \"fdb8314d-7223-4926-b551-8bb1701052fd\") " pod="openshift-dns/node-resolver-btdbz"
Sep 15 16:26:24.892873 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892469    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-etc-sysctl-d\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.892873 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892493    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4893f9dd-7ff1-419a-9e27-c12efdb8285d-konnectivity-ca\") pod \"konnectivity-agent-s7d5t\" (UID: \"4893f9dd-7ff1-419a-9e27-c12efdb8285d\") " pod="kube-system/konnectivity-agent-s7d5t"
Sep 15 16:26:24.892873 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892493    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3c332c22-48cd-4420-9a35-e40f98ec4344-socket-dir\") pod \"aws-ebs-csi-driver-node-rw6mp\" (UID: \"3c332c22-48cd-4420-9a35-e40f98ec4344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rw6mp"
Sep 15 16:26:24.892873 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892517    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-cnibin\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.892873 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892548    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-run-systemd\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.892873 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892573    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f89021a6-bc8a-4f31-a76c-87c39f332c20-serviceca\") pod \"node-ca-8zk2f\" (UID: \"f89021a6-bc8a-4f31-a76c-87c39f332c20\") " pod="openshift-image-registry/node-ca-8zk2f"
Sep 15 16:26:24.892873 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892588    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a83b81c-a949-41e4-9e47-e7305b24d4a7-tmp\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.892873 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892631    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-run-openvswitch\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.892873 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892637    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-run-systemd\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.893573 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892652    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d1e544d2-21e5-4abd-82d9-33a82470f13c-original-pull-secret\") pod \"global-pull-secret-syncer-9hbcw\" (UID: \"d1e544d2-21e5-4abd-82d9-33a82470f13c\") " pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:24.893573 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892667    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-multus-socket-dir-parent\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.893573 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892666    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/44ca88f0-c149-443e-a2d1-a30964775818-env-overrides\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.893573 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892670    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-run-openvswitch\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.893573 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892691    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-host-var-lib-kubelet\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.893573 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892732    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-etc-modprobe-d\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.893573 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892692    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bf4e438a-fbc7-401f-b97c-bbb7bc4a0495-iptables-alerter-script\") pod \"iptables-alerter-cs7tq\" (UID: \"bf4e438a-fbc7-401f-b97c-bbb7bc4a0495\") " pod="openshift-network-operator/iptables-alerter-cs7tq"
Sep 15 16:26:24.893573 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892802    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-run\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.893573 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892828    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-run-ovn\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.893573 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892852    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-host-run-ovn-kubernetes\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.893573 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892875    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-lib-modules\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.893573 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892891    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-host-run-ovn-kubernetes\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.893573 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892877    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-run-ovn\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.893573 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892897    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-452xc\" (UniqueName: \"kubernetes.io/projected/7a83b81c-a949-41e4-9e47-e7305b24d4a7-kube-api-access-452xc\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.893573 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892937    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bf4e438a-fbc7-401f-b97c-bbb7bc4a0495-host-slash\") pod \"iptables-alerter-cs7tq\" (UID: \"bf4e438a-fbc7-401f-b97c-bbb7bc4a0495\") " pod="openshift-network-operator/iptables-alerter-cs7tq"
Sep 15 16:26:24.893573 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892959    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/44ca88f0-c149-443e-a2d1-a30964775818-ovnkube-script-lib\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.893573 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892982    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b17d9664-4ce9-449a-8826-af717fd47f97-cni-binary-copy\") pod \"multus-additional-cni-plugins-cvgk4\" (UID: \"b17d9664-4ce9-449a-8826-af717fd47f97\") " pod="openshift-multus/multus-additional-cni-plugins-cvgk4"
Sep 15 16:26:24.894174 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.892989    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bf4e438a-fbc7-401f-b97c-bbb7bc4a0495-host-slash\") pod \"iptables-alerter-cs7tq\" (UID: \"bf4e438a-fbc7-401f-b97c-bbb7bc4a0495\") " pod="openshift-network-operator/iptables-alerter-cs7tq"
Sep 15 16:26:24.894174 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893004    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3c332c22-48cd-4420-9a35-e40f98ec4344-etc-selinux\") pod \"aws-ebs-csi-driver-node-rw6mp\" (UID: \"3c332c22-48cd-4420-9a35-e40f98ec4344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rw6mp"
Sep 15 16:26:24.894174 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893028    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d768ac0-1487-4850-bd9b-c66f73533e97-metrics-certs\") pod \"network-metrics-daemon-txzqk\" (UID: \"1d768ac0-1487-4850-bd9b-c66f73533e97\") " pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:26:24.894174 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893098    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3c332c22-48cd-4420-9a35-e40f98ec4344-etc-selinux\") pod \"aws-ebs-csi-driver-node-rw6mp\" (UID: \"3c332c22-48cd-4420-9a35-e40f98ec4344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rw6mp"
Sep 15 16:26:24.894174 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:24.893120    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 16:26:24.894174 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893144    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-etc-sysconfig\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.894174 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893165    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3c332c22-48cd-4420-9a35-e40f98ec4344-registration-dir\") pod \"aws-ebs-csi-driver-node-rw6mp\" (UID: \"3c332c22-48cd-4420-9a35-e40f98ec4344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rw6mp"
Sep 15 16:26:24.894174 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893184    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f89021a6-bc8a-4f31-a76c-87c39f332c20-host\") pod \"node-ca-8zk2f\" (UID: \"f89021a6-bc8a-4f31-a76c-87c39f332c20\") " pod="openshift-image-registry/node-ca-8zk2f"
Sep 15 16:26:24.894174 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893216    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-host-slash\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.894174 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893239    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3c332c22-48cd-4420-9a35-e40f98ec4344-registration-dir\") pod \"aws-ebs-csi-driver-node-rw6mp\" (UID: \"3c332c22-48cd-4420-9a35-e40f98ec4344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rw6mp"
Sep 15 16:26:24.894174 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893242    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-var-lib-openvswitch\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.894174 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:24.893283    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d768ac0-1487-4850-bd9b-c66f73533e97-metrics-certs podName:1d768ac0-1487-4850-bd9b-c66f73533e97 nodeName:}" failed. No retries permitted until 2026-09-15 16:26:25.393248209 +0000 UTC m=+2.022902564 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d768ac0-1487-4850-bd9b-c66f73533e97-metrics-certs") pod "network-metrics-daemon-txzqk" (UID: "1d768ac0-1487-4850-bd9b-c66f73533e97") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 16:26:24.894174 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893285    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-var-lib-openvswitch\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.894174 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893290    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-host-slash\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.894174 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893321    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-log-socket\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.894174 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893345    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.894174 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893363    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d1e544d2-21e5-4abd-82d9-33a82470f13c-kubelet-config\") pod \"global-pull-secret-syncer-9hbcw\" (UID: \"d1e544d2-21e5-4abd-82d9-33a82470f13c\") " pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:24.894743 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893381    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-host-run-k8s-cni-cncf-io\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.894743 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893401    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-host-run-netns\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.894743 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893402    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.894743 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893419    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-hostroot\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.894743 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893421    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-log-socket\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.894743 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893435    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5h7q\" (UniqueName: \"kubernetes.io/projected/ad430be2-7c5f-4402-a5a7-c8b4359b8c02-kube-api-access-r5h7q\") pod \"network-check-target-sqdn2\" (UID: \"ad430be2-7c5f-4402-a5a7-c8b4359b8c02\") " pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:26:24.894743 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893441    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/44ca88f0-c149-443e-a2d1-a30964775818-ovnkube-script-lib\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.894743 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893454    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5dw9\" (UniqueName: \"kubernetes.io/projected/f89021a6-bc8a-4f31-a76c-87c39f332c20-kube-api-access-h5dw9\") pod \"node-ca-8zk2f\" (UID: \"f89021a6-bc8a-4f31-a76c-87c39f332c20\") " pod="openshift-image-registry/node-ca-8zk2f"
Sep 15 16:26:24.894743 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893441    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b17d9664-4ce9-449a-8826-af717fd47f97-cni-binary-copy\") pod \"multus-additional-cni-plugins-cvgk4\" (UID: \"b17d9664-4ce9-449a-8826-af717fd47f97\") " pod="openshift-multus/multus-additional-cni-plugins-cvgk4"
Sep 15 16:26:24.894743 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893471    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-system-cni-dir\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.894743 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893493    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-var-lib-kubelet\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.894743 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893514    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-systemd-units\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.894743 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893535    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sbz4\" (UniqueName: \"kubernetes.io/projected/44ca88f0-c149-443e-a2d1-a30964775818-kube-api-access-2sbz4\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.894743 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893552    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b17d9664-4ce9-449a-8826-af717fd47f97-system-cni-dir\") pod \"multus-additional-cni-plugins-cvgk4\" (UID: \"b17d9664-4ce9-449a-8826-af717fd47f97\") " pod="openshift-multus/multus-additional-cni-plugins-cvgk4"
Sep 15 16:26:24.894743 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893589    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-systemd-units\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.894743 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893594    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b17d9664-4ce9-449a-8826-af717fd47f97-cnibin\") pod \"multus-additional-cni-plugins-cvgk4\" (UID: \"b17d9664-4ce9-449a-8826-af717fd47f97\") " pod="openshift-multus/multus-additional-cni-plugins-cvgk4"
Sep 15 16:26:24.894743 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893615    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b17d9664-4ce9-449a-8826-af717fd47f97-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cvgk4\" (UID: \"b17d9664-4ce9-449a-8826-af717fd47f97\") " pod="openshift-multus/multus-additional-cni-plugins-cvgk4"
Sep 15 16:26:24.895258 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893621    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b17d9664-4ce9-449a-8826-af717fd47f97-cnibin\") pod \"multus-additional-cni-plugins-cvgk4\" (UID: \"b17d9664-4ce9-449a-8826-af717fd47f97\") " pod="openshift-multus/multus-additional-cni-plugins-cvgk4"
Sep 15 16:26:24.895258 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893633    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-multus-cni-dir\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.895258 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893647    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-etc-kubernetes\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.895258 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893649    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b17d9664-4ce9-449a-8826-af717fd47f97-system-cni-dir\") pod \"multus-additional-cni-plugins-cvgk4\" (UID: \"b17d9664-4ce9-449a-8826-af717fd47f97\") " pod="openshift-multus/multus-additional-cni-plugins-cvgk4"
Sep 15 16:26:24.895258 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893679    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-etc-systemd\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.895258 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893713    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mflr\" (UniqueName: \"kubernetes.io/projected/bf4e438a-fbc7-401f-b97c-bbb7bc4a0495-kube-api-access-9mflr\") pod \"iptables-alerter-cs7tq\" (UID: \"bf4e438a-fbc7-401f-b97c-bbb7bc4a0495\") " pod="openshift-network-operator/iptables-alerter-cs7tq"
Sep 15 16:26:24.895258 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893733    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d1e544d2-21e5-4abd-82d9-33a82470f13c-dbus\") pod \"global-pull-secret-syncer-9hbcw\" (UID: \"d1e544d2-21e5-4abd-82d9-33a82470f13c\") " pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:24.895258 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893826    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gft82\" (UniqueName: \"kubernetes.io/projected/fdb8314d-7223-4926-b551-8bb1701052fd-kube-api-access-gft82\") pod \"node-resolver-btdbz\" (UID: \"fdb8314d-7223-4926-b551-8bb1701052fd\") " pod="openshift-dns/node-resolver-btdbz"
Sep 15 16:26:24.895258 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893855    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-cni-binary-copy\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.895258 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893881    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-host-var-lib-cni-bin\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.895258 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893904    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-etc-sysctl-conf\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.895258 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893930    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/44ca88f0-c149-443e-a2d1-a30964775818-ovn-node-metrics-cert\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.895258 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.893978    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-host-kubelet\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.895258 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.894004    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-host-kubelet\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.895258 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.894029    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-etc-openvswitch\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.895258 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.894069    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/44ca88f0-c149-443e-a2d1-a30964775818-ovnkube-config\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.895258 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.894086    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qc2kp\" (UniqueName: \"kubernetes.io/projected/b17d9664-4ce9-449a-8826-af717fd47f97-kube-api-access-qc2kp\") pod \"multus-additional-cni-plugins-cvgk4\" (UID: \"b17d9664-4ce9-449a-8826-af717fd47f97\") " pod="openshift-multus/multus-additional-cni-plugins-cvgk4"
Sep 15 16:26:24.895711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.894087    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b17d9664-4ce9-449a-8826-af717fd47f97-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cvgk4\" (UID: \"b17d9664-4ce9-449a-8826-af717fd47f97\") " pod="openshift-multus/multus-additional-cni-plugins-cvgk4"
Sep 15 16:26:24.895711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.894102    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4893f9dd-7ff1-419a-9e27-c12efdb8285d-agent-certs\") pod \"konnectivity-agent-s7d5t\" (UID: \"4893f9dd-7ff1-419a-9e27-c12efdb8285d\") " pod="kube-system/konnectivity-agent-s7d5t"
Sep 15 16:26:24.895711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.894085    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44ca88f0-c149-443e-a2d1-a30964775818-etc-openvswitch\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.895711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.894139    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b17d9664-4ce9-449a-8826-af717fd47f97-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cvgk4\" (UID: \"b17d9664-4ce9-449a-8826-af717fd47f97\") " pod="openshift-multus/multus-additional-cni-plugins-cvgk4"
Sep 15 16:26:24.895711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.894240    2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 15 16:26:24.895711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.894300    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b17d9664-4ce9-449a-8826-af717fd47f97-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cvgk4\" (UID: \"b17d9664-4ce9-449a-8826-af717fd47f97\") " pod="openshift-multus/multus-additional-cni-plugins-cvgk4"
Sep 15 16:26:24.895711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.894453    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/44ca88f0-c149-443e-a2d1-a30964775818-ovnkube-config\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.897134 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.897117    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/44ca88f0-c149-443e-a2d1-a30964775818-ovn-node-metrics-cert\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.901630 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.901587    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgj9r\" (UniqueName: \"kubernetes.io/projected/3c332c22-48cd-4420-9a35-e40f98ec4344-kube-api-access-wgj9r\") pod \"aws-ebs-csi-driver-node-rw6mp\" (UID: \"3c332c22-48cd-4420-9a35-e40f98ec4344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rw6mp"
Sep 15 16:26:24.901910 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.901893    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wwqd\" (UniqueName: \"kubernetes.io/projected/1d768ac0-1487-4850-bd9b-c66f73533e97-kube-api-access-6wwqd\") pod \"network-metrics-daemon-txzqk\" (UID: \"1d768ac0-1487-4850-bd9b-c66f73533e97\") " pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:26:24.901965 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.901946    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mflr\" (UniqueName: \"kubernetes.io/projected/bf4e438a-fbc7-401f-b97c-bbb7bc4a0495-kube-api-access-9mflr\") pod \"iptables-alerter-cs7tq\" (UID: \"bf4e438a-fbc7-401f-b97c-bbb7bc4a0495\") " pod="openshift-network-operator/iptables-alerter-cs7tq"
Sep 15 16:26:24.902592 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.902568    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sbz4\" (UniqueName: \"kubernetes.io/projected/44ca88f0-c149-443e-a2d1-a30964775818-kube-api-access-2sbz4\") pod \"ovnkube-node-b4rdn\" (UID: \"44ca88f0-c149-443e-a2d1-a30964775818\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:24.902694 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.902678    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qc2kp\" (UniqueName: \"kubernetes.io/projected/b17d9664-4ce9-449a-8826-af717fd47f97-kube-api-access-qc2kp\") pod \"multus-additional-cni-plugins-cvgk4\" (UID: \"b17d9664-4ce9-449a-8826-af717fd47f97\") " pod="openshift-multus/multus-additional-cni-plugins-cvgk4"
Sep 15 16:26:24.964997 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:24.964976    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod269bca4735829fdfafb55c4d16c2e698.slice/crio-083ac553a8b051018063c58b7da71cb90f30c3d181021c026b692402abfa74ba WatchSource:0}: Error finding container 083ac553a8b051018063c58b7da71cb90f30c3d181021c026b692402abfa74ba: Status 404 returned error can't find the container with id 083ac553a8b051018063c58b7da71cb90f30c3d181021c026b692402abfa74ba
Sep 15 16:26:24.969386 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.969370    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 16:26:24.995369 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995347    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d1e544d2-21e5-4abd-82d9-33a82470f13c-kubelet-config\") pod \"global-pull-secret-syncer-9hbcw\" (UID: \"d1e544d2-21e5-4abd-82d9-33a82470f13c\") " pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:24.995446 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995370    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-host-run-k8s-cni-cncf-io\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.995446 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995388    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-host-run-netns\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.995446 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995410    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-hostroot\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.995446 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995437    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5h7q\" (UniqueName: \"kubernetes.io/projected/ad430be2-7c5f-4402-a5a7-c8b4359b8c02-kube-api-access-r5h7q\") pod \"network-check-target-sqdn2\" (UID: \"ad430be2-7c5f-4402-a5a7-c8b4359b8c02\") " pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:26:24.995639 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995450    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-host-run-k8s-cni-cncf-io\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.995639 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995461    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-host-run-netns\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.995639 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995473    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-hostroot\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.995639 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995463    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5dw9\" (UniqueName: \"kubernetes.io/projected/f89021a6-bc8a-4f31-a76c-87c39f332c20-kube-api-access-h5dw9\") pod \"node-ca-8zk2f\" (UID: \"f89021a6-bc8a-4f31-a76c-87c39f332c20\") " pod="openshift-image-registry/node-ca-8zk2f"
Sep 15 16:26:24.995639 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995512    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-system-cni-dir\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.995639 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995527    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-var-lib-kubelet\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.995639 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995489    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d1e544d2-21e5-4abd-82d9-33a82470f13c-kubelet-config\") pod \"global-pull-secret-syncer-9hbcw\" (UID: \"d1e544d2-21e5-4abd-82d9-33a82470f13c\") " pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:24.995639 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995543    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-multus-cni-dir\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.995639 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995576    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-etc-kubernetes\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.995639 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995583    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-system-cni-dir\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.995639 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995587    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-var-lib-kubelet\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.995639 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995599    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-etc-systemd\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.995639 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995626    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-etc-kubernetes\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.995639 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995642    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-etc-systemd\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.995639 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995646    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d1e544d2-21e5-4abd-82d9-33a82470f13c-dbus\") pod \"global-pull-secret-syncer-9hbcw\" (UID: \"d1e544d2-21e5-4abd-82d9-33a82470f13c\") " pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:24.996278 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995667    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-multus-cni-dir\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.996278 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995672    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gft82\" (UniqueName: \"kubernetes.io/projected/fdb8314d-7223-4926-b551-8bb1701052fd-kube-api-access-gft82\") pod \"node-resolver-btdbz\" (UID: \"fdb8314d-7223-4926-b551-8bb1701052fd\") " pod="openshift-dns/node-resolver-btdbz"
Sep 15 16:26:24.996278 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995691    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d1e544d2-21e5-4abd-82d9-33a82470f13c-dbus\") pod \"global-pull-secret-syncer-9hbcw\" (UID: \"d1e544d2-21e5-4abd-82d9-33a82470f13c\") " pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:24.996278 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995695    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-cni-binary-copy\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.996278 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995726    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-host-var-lib-cni-bin\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.996278 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995749    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-etc-sysctl-conf\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.996278 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995771    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-host-var-lib-cni-bin\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.996278 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995809    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4893f9dd-7ff1-419a-9e27-c12efdb8285d-agent-certs\") pod \"konnectivity-agent-s7d5t\" (UID: \"4893f9dd-7ff1-419a-9e27-c12efdb8285d\") " pod="kube-system/konnectivity-agent-s7d5t"
Sep 15 16:26:24.996278 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995837    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-multus-conf-dir\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.996278 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995859    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdb8314d-7223-4926-b551-8bb1701052fd-tmp-dir\") pod \"node-resolver-btdbz\" (UID: \"fdb8314d-7223-4926-b551-8bb1701052fd\") " pod="openshift-dns/node-resolver-btdbz"
Sep 15 16:26:24.996278 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995880    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-multus-daemon-config\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.996278 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995892    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-etc-sysctl-conf\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.996278 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995902    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-host-run-multus-certs\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.996278 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995924    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-etc-kubernetes\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.996278 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995939    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-host\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.996278 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995953    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-host-var-lib-cni-multus\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.996278 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.995976    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv6bf\" (UniqueName: \"kubernetes.io/projected/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-kube-api-access-sv6bf\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.996278 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996005    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-multus-conf-dir\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.997377 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996018    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-sys\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.997377 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996039    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7a83b81c-a949-41e4-9e47-e7305b24d4a7-etc-tuned\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.997377 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996040    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-host-var-lib-cni-multus\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.997377 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996054    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-host-run-multus-certs\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.997377 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996067    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fdb8314d-7223-4926-b551-8bb1701052fd-hosts-file\") pod \"node-resolver-btdbz\" (UID: \"fdb8314d-7223-4926-b551-8bb1701052fd\") " pod="openshift-dns/node-resolver-btdbz"
Sep 15 16:26:24.997377 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996010    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-etc-kubernetes\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.997377 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996089    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-etc-sysctl-d\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.997377 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996108    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-host\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.997377 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996131    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4893f9dd-7ff1-419a-9e27-c12efdb8285d-konnectivity-ca\") pod \"konnectivity-agent-s7d5t\" (UID: \"4893f9dd-7ff1-419a-9e27-c12efdb8285d\") " pod="kube-system/konnectivity-agent-s7d5t"
Sep 15 16:26:24.997377 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996152    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-cnibin\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.997377 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996161    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdb8314d-7223-4926-b551-8bb1701052fd-tmp-dir\") pod \"node-resolver-btdbz\" (UID: \"fdb8314d-7223-4926-b551-8bb1701052fd\") " pod="openshift-dns/node-resolver-btdbz"
Sep 15 16:26:24.997377 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996175    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f89021a6-bc8a-4f31-a76c-87c39f332c20-serviceca\") pod \"node-ca-8zk2f\" (UID: \"f89021a6-bc8a-4f31-a76c-87c39f332c20\") " pod="openshift-image-registry/node-ca-8zk2f"
Sep 15 16:26:24.997377 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996205    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a83b81c-a949-41e4-9e47-e7305b24d4a7-tmp\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.997377 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996246    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d1e544d2-21e5-4abd-82d9-33a82470f13c-original-pull-secret\") pod \"global-pull-secret-syncer-9hbcw\" (UID: \"d1e544d2-21e5-4abd-82d9-33a82470f13c\") " pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:24.997377 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996282    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-multus-socket-dir-parent\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.997377 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996310    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-host-var-lib-kubelet\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.997377 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996321    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fdb8314d-7223-4926-b551-8bb1701052fd-hosts-file\") pod \"node-resolver-btdbz\" (UID: \"fdb8314d-7223-4926-b551-8bb1701052fd\") " pod="openshift-dns/node-resolver-btdbz"
Sep 15 16:26:24.997377 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:24.996332    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 16:26:24.997377 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996336    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-etc-modprobe-d\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.998232 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996341    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-multus-daemon-config\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.998232 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996359    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-run\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.998232 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996283    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-cnibin\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.998232 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996249    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-cni-binary-copy\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.998232 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:24.996388    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e544d2-21e5-4abd-82d9-33a82470f13c-original-pull-secret podName:d1e544d2-21e5-4abd-82d9-33a82470f13c nodeName:}" failed. No retries permitted until 2026-09-15 16:26:25.496372048 +0000 UTC m=+2.126026385 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d1e544d2-21e5-4abd-82d9-33a82470f13c-original-pull-secret") pod "global-pull-secret-syncer-9hbcw" (UID: "d1e544d2-21e5-4abd-82d9-33a82470f13c") : object "kube-system"/"original-pull-secret" not registered
Sep 15 16:26:24.998232 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996407    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-lib-modules\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.998232 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996417    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-run\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.998232 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996430    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-multus-socket-dir-parent\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.998232 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996434    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-452xc\" (UniqueName: \"kubernetes.io/projected/7a83b81c-a949-41e4-9e47-e7305b24d4a7-kube-api-access-452xc\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.998232 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996468    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-host-var-lib-kubelet\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:24.998232 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996476    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-etc-sysconfig\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.998232 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996500    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f89021a6-bc8a-4f31-a76c-87c39f332c20-host\") pod \"node-ca-8zk2f\" (UID: \"f89021a6-bc8a-4f31-a76c-87c39f332c20\") " pod="openshift-image-registry/node-ca-8zk2f"
Sep 15 16:26:24.998232 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996506    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-etc-modprobe-d\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.998232 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996556    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-etc-sysctl-d\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.998232 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996538    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-sys\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.998232 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996607    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f89021a6-bc8a-4f31-a76c-87c39f332c20-host\") pod \"node-ca-8zk2f\" (UID: \"f89021a6-bc8a-4f31-a76c-87c39f332c20\") " pod="openshift-image-registry/node-ca-8zk2f"
Sep 15 16:26:24.998232 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996610    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-etc-sysconfig\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.998753 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996676    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7a83b81c-a949-41e4-9e47-e7305b24d4a7-lib-modules\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.998753 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.996853    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f89021a6-bc8a-4f31-a76c-87c39f332c20-serviceca\") pod \"node-ca-8zk2f\" (UID: \"f89021a6-bc8a-4f31-a76c-87c39f332c20\") " pod="openshift-image-registry/node-ca-8zk2f"
Sep 15 16:26:24.998753 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.997470    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4893f9dd-7ff1-419a-9e27-c12efdb8285d-konnectivity-ca\") pod \"konnectivity-agent-s7d5t\" (UID: \"4893f9dd-7ff1-419a-9e27-c12efdb8285d\") " pod="kube-system/konnectivity-agent-s7d5t"
Sep 15 16:26:24.998753 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.998183    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7a83b81c-a949-41e4-9e47-e7305b24d4a7-etc-tuned\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:24.998753 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.998473    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4893f9dd-7ff1-419a-9e27-c12efdb8285d-agent-certs\") pod \"konnectivity-agent-s7d5t\" (UID: \"4893f9dd-7ff1-419a-9e27-c12efdb8285d\") " pod="kube-system/konnectivity-agent-s7d5t"
Sep 15 16:26:24.998753 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:24.998515    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a83b81c-a949-41e4-9e47-e7305b24d4a7-tmp\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:25.005193 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:25.005179    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 16:26:25.005193 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:25.005195    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 16:26:25.005293 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:25.005203    2579 projected.go:196] Error preparing data for projected volume kube-api-access-r5h7q for pod openshift-network-diagnostics/network-check-target-sqdn2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 16:26:25.005293 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:25.005237    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ad430be2-7c5f-4402-a5a7-c8b4359b8c02-kube-api-access-r5h7q podName:ad430be2-7c5f-4402-a5a7-c8b4359b8c02 nodeName:}" failed. No retries permitted until 2026-09-15 16:26:25.505226717 +0000 UTC m=+2.134881070 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-r5h7q" (UniqueName: "kubernetes.io/projected/ad430be2-7c5f-4402-a5a7-c8b4359b8c02-kube-api-access-r5h7q") pod "network-check-target-sqdn2" (UID: "ad430be2-7c5f-4402-a5a7-c8b4359b8c02") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 16:26:25.006947 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.006920    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv6bf\" (UniqueName: \"kubernetes.io/projected/ba7071d8-d459-4c22-ae2f-1ced3f8871ce-kube-api-access-sv6bf\") pod \"multus-dqm7s\" (UID: \"ba7071d8-d459-4c22-ae2f-1ced3f8871ce\") " pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:25.007215 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.007198    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-452xc\" (UniqueName: \"kubernetes.io/projected/7a83b81c-a949-41e4-9e47-e7305b24d4a7-kube-api-access-452xc\") pod \"tuned-zr9w5\" (UID: \"7a83b81c-a949-41e4-9e47-e7305b24d4a7\") " pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:25.007367 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.007351    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5dw9\" (UniqueName: \"kubernetes.io/projected/f89021a6-bc8a-4f31-a76c-87c39f332c20-kube-api-access-h5dw9\") pod \"node-ca-8zk2f\" (UID: \"f89021a6-bc8a-4f31-a76c-87c39f332c20\") " pod="openshift-image-registry/node-ca-8zk2f"
Sep 15 16:26:25.007367 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.007361    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gft82\" (UniqueName: \"kubernetes.io/projected/fdb8314d-7223-4926-b551-8bb1701052fd-kube-api-access-gft82\") pod \"node-resolver-btdbz\" (UID: \"fdb8314d-7223-4926-b551-8bb1701052fd\") " pod="openshift-dns/node-resolver-btdbz"
Sep 15 16:26:25.106704 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.106664    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cvgk4"
Sep 15 16:26:25.115177 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:25.115154    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb17d9664_4ce9_449a_8826_af717fd47f97.slice/crio-54c2ede766b518d0c7a4b0a1b2bc50badcdbdf844348052b2accc1ef569b30cc WatchSource:0}: Error finding container 54c2ede766b518d0c7a4b0a1b2bc50badcdbdf844348052b2accc1ef569b30cc: Status 404 returned error can't find the container with id 54c2ede766b518d0c7a4b0a1b2bc50badcdbdf844348052b2accc1ef569b30cc
Sep 15 16:26:25.121941 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.121922    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rw6mp"
Sep 15 16:26:25.129923 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:25.129900    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c332c22_48cd_4420_9a35_e40f98ec4344.slice/crio-4a0bd09319397baa7d7d709d3d921bfd4b4f282f9aa494260995a3b68deef543 WatchSource:0}: Error finding container 4a0bd09319397baa7d7d709d3d921bfd4b4f282f9aa494260995a3b68deef543: Status 404 returned error can't find the container with id 4a0bd09319397baa7d7d709d3d921bfd4b4f282f9aa494260995a3b68deef543
Sep 15 16:26:25.136290 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.136273    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cs7tq"
Sep 15 16:26:25.143731 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:25.143714    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf4e438a_fbc7_401f_b97c_bbb7bc4a0495.slice/crio-826d9e207aec2b322158fe16410fa3d5fc345dd14546acdf4d570ecd3ced46b1 WatchSource:0}: Error finding container 826d9e207aec2b322158fe16410fa3d5fc345dd14546acdf4d570ecd3ced46b1: Status 404 returned error can't find the container with id 826d9e207aec2b322158fe16410fa3d5fc345dd14546acdf4d570ecd3ced46b1
Sep 15 16:26:25.150524 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.150508    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:25.157705 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:25.157686    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44ca88f0_c149_443e_a2d1_a30964775818.slice/crio-dbb4be4a114933d11d75512cf3aedd992afbfd7fcfe1b2900044ae021481505f WatchSource:0}: Error finding container dbb4be4a114933d11d75512cf3aedd992afbfd7fcfe1b2900044ae021481505f: Status 404 returned error can't find the container with id dbb4be4a114933d11d75512cf3aedd992afbfd7fcfe1b2900044ae021481505f
Sep 15 16:26:25.168298 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.168283    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-s7d5t"
Sep 15 16:26:25.176446 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:25.176429    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4893f9dd_7ff1_419a_9e27_c12efdb8285d.slice/crio-ab130ce553129aa65c7d0331fd0543aef3f142a109a27043fa623316570f3e90 WatchSource:0}: Error finding container ab130ce553129aa65c7d0331fd0543aef3f142a109a27043fa623316570f3e90: Status 404 returned error can't find the container with id ab130ce553129aa65c7d0331fd0543aef3f142a109a27043fa623316570f3e90
Sep 15 16:26:25.188076 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.188059    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8zk2f"
Sep 15 16:26:25.195751 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:25.195737    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf89021a6_bc8a_4f31_a76c_87c39f332c20.slice/crio-9dfb27a749d9371f6b739fd5d66db621c1a5e51ae6e11da7e9d184e8931fff9e WatchSource:0}: Error finding container 9dfb27a749d9371f6b739fd5d66db621c1a5e51ae6e11da7e9d184e8931fff9e: Status 404 returned error can't find the container with id 9dfb27a749d9371f6b739fd5d66db621c1a5e51ae6e11da7e9d184e8931fff9e
Sep 15 16:26:25.209050 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.209033    2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 16:26:25.214220 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.214206    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-btdbz"
Sep 15 16:26:25.221333 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:25.221313    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdb8314d_7223_4926_b551_8bb1701052fd.slice/crio-f5e89052b431d0ceb59c96bbe1c67d81cb241b15565d86d9b2e76368f10d70ee WatchSource:0}: Error finding container f5e89052b431d0ceb59c96bbe1c67d81cb241b15565d86d9b2e76368f10d70ee: Status 404 returned error can't find the container with id f5e89052b431d0ceb59c96bbe1c67d81cb241b15565d86d9b2e76368f10d70ee
Sep 15 16:26:25.231071 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.231042    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dqm7s"
Sep 15 16:26:25.235245 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.235226    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zr9w5"
Sep 15 16:26:25.238227 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:25.238212    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba7071d8_d459_4c22_ae2f_1ced3f8871ce.slice/crio-88f426b01911c3321f56aa45d1f607aaf63c1b884126d7464f0af0248b280dc1 WatchSource:0}: Error finding container 88f426b01911c3321f56aa45d1f607aaf63c1b884126d7464f0af0248b280dc1: Status 404 returned error can't find the container with id 88f426b01911c3321f56aa45d1f607aaf63c1b884126d7464f0af0248b280dc1
Sep 15 16:26:25.242239 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:25.242222    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a83b81c_a949_41e4_9e47_e7305b24d4a7.slice/crio-423a1b08fe28f0abf1bf24f5c6fa54f1414186071bede706d9dedda708838d05 WatchSource:0}: Error finding container 423a1b08fe28f0abf1bf24f5c6fa54f1414186071bede706d9dedda708838d05: Status 404 returned error can't find the container with id 423a1b08fe28f0abf1bf24f5c6fa54f1414186071bede706d9dedda708838d05
Sep 15 16:26:25.400311 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.400265    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d768ac0-1487-4850-bd9b-c66f73533e97-metrics-certs\") pod \"network-metrics-daemon-txzqk\" (UID: \"1d768ac0-1487-4850-bd9b-c66f73533e97\") " pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:26:25.400382 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:25.400354    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 16:26:25.400418 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:25.400396    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d768ac0-1487-4850-bd9b-c66f73533e97-metrics-certs podName:1d768ac0-1487-4850-bd9b-c66f73533e97 nodeName:}" failed. No retries permitted until 2026-09-15 16:26:26.400381314 +0000 UTC m=+3.030035668 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d768ac0-1487-4850-bd9b-c66f73533e97-metrics-certs") pod "network-metrics-daemon-txzqk" (UID: "1d768ac0-1487-4850-bd9b-c66f73533e97") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 16:26:25.501491 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.501466    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d1e544d2-21e5-4abd-82d9-33a82470f13c-original-pull-secret\") pod \"global-pull-secret-syncer-9hbcw\" (UID: \"d1e544d2-21e5-4abd-82d9-33a82470f13c\") " pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:25.501607 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:25.501586    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 16:26:25.501660 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:25.501651    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e544d2-21e5-4abd-82d9-33a82470f13c-original-pull-secret podName:d1e544d2-21e5-4abd-82d9-33a82470f13c nodeName:}" failed. No retries permitted until 2026-09-15 16:26:26.501635211 +0000 UTC m=+3.131289550 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d1e544d2-21e5-4abd-82d9-33a82470f13c-original-pull-secret") pod "global-pull-secret-syncer-9hbcw" (UID: "d1e544d2-21e5-4abd-82d9-33a82470f13c") : object "kube-system"/"original-pull-secret" not registered
Sep 15 16:26:25.575761 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.575738    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 16:26:25.602284 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.602042    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5h7q\" (UniqueName: \"kubernetes.io/projected/ad430be2-7c5f-4402-a5a7-c8b4359b8c02-kube-api-access-r5h7q\") pod \"network-check-target-sqdn2\" (UID: \"ad430be2-7c5f-4402-a5a7-c8b4359b8c02\") " pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:26:25.602284 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:25.602205    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 16:26:25.602284 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:25.602223    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 16:26:25.602284 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:25.602233    2579 projected.go:196] Error preparing data for projected volume kube-api-access-r5h7q for pod openshift-network-diagnostics/network-check-target-sqdn2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 16:26:25.602284 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:25.602274    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ad430be2-7c5f-4402-a5a7-c8b4359b8c02-kube-api-access-r5h7q podName:ad430be2-7c5f-4402-a5a7-c8b4359b8c02 nodeName:}" failed. No retries permitted until 2026-09-15 16:26:26.602260467 +0000 UTC m=+3.231914801 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-r5h7q" (UniqueName: "kubernetes.io/projected/ad430be2-7c5f-4402-a5a7-c8b4359b8c02-kube-api-access-r5h7q") pod "network-check-target-sqdn2" (UID: "ad430be2-7c5f-4402-a5a7-c8b4359b8c02") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 16:26:25.850187 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.850124    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 16:21:24 +0000 UTC" deadline="2028-05-11 07:14:35.12535303 +0000 UTC"
Sep 15 16:26:25.850187 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.850185    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14486h48m9.275171496s"
Sep 15 16:26:25.903804 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.903093    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:25.903804 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:25.903213    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hbcw" podUID="d1e544d2-21e5-4abd-82d9-33a82470f13c"
Sep 15 16:26:25.911980 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.911925    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-s7d5t" event={"ID":"4893f9dd-7ff1-419a-9e27-c12efdb8285d","Type":"ContainerStarted","Data":"ab130ce553129aa65c7d0331fd0543aef3f142a109a27043fa623316570f3e90"}
Sep 15 16:26:25.914588 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.914565    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn" event={"ID":"44ca88f0-c149-443e-a2d1-a30964775818","Type":"ContainerStarted","Data":"dbb4be4a114933d11d75512cf3aedd992afbfd7fcfe1b2900044ae021481505f"}
Sep 15 16:26:25.916470 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.916444    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rw6mp" event={"ID":"3c332c22-48cd-4420-9a35-e40f98ec4344","Type":"ContainerStarted","Data":"4a0bd09319397baa7d7d709d3d921bfd4b4f282f9aa494260995a3b68deef543"}
Sep 15 16:26:25.921824 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.921796    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-35.ec2.internal" event={"ID":"269bca4735829fdfafb55c4d16c2e698","Type":"ContainerStarted","Data":"083ac553a8b051018063c58b7da71cb90f30c3d181021c026b692402abfa74ba"}
Sep 15 16:26:25.925083 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.925060    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zr9w5" event={"ID":"7a83b81c-a949-41e4-9e47-e7305b24d4a7","Type":"ContainerStarted","Data":"423a1b08fe28f0abf1bf24f5c6fa54f1414186071bede706d9dedda708838d05"}
Sep 15 16:26:25.928337 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.928291    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dqm7s" event={"ID":"ba7071d8-d459-4c22-ae2f-1ced3f8871ce","Type":"ContainerStarted","Data":"88f426b01911c3321f56aa45d1f607aaf63c1b884126d7464f0af0248b280dc1"}
Sep 15 16:26:25.930932 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.930910    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-btdbz" event={"ID":"fdb8314d-7223-4926-b551-8bb1701052fd","Type":"ContainerStarted","Data":"f5e89052b431d0ceb59c96bbe1c67d81cb241b15565d86d9b2e76368f10d70ee"}
Sep 15 16:26:25.934672 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.934648    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8zk2f" event={"ID":"f89021a6-bc8a-4f31-a76c-87c39f332c20","Type":"ContainerStarted","Data":"9dfb27a749d9371f6b739fd5d66db621c1a5e51ae6e11da7e9d184e8931fff9e"}
Sep 15 16:26:25.937636 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.937613    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cs7tq" event={"ID":"bf4e438a-fbc7-401f-b97c-bbb7bc4a0495","Type":"ContainerStarted","Data":"826d9e207aec2b322158fe16410fa3d5fc345dd14546acdf4d570ecd3ced46b1"}
Sep 15 16:26:25.940098 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:25.940073    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cvgk4" event={"ID":"b17d9664-4ce9-449a-8826-af717fd47f97","Type":"ContainerStarted","Data":"54c2ede766b518d0c7a4b0a1b2bc50badcdbdf844348052b2accc1ef569b30cc"}
Sep 15 16:26:26.240234 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:26.240210    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 16:26:26.408366 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:26.407855    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d768ac0-1487-4850-bd9b-c66f73533e97-metrics-certs\") pod \"network-metrics-daemon-txzqk\" (UID: \"1d768ac0-1487-4850-bd9b-c66f73533e97\") " pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:26:26.408366 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:26.408009    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 16:26:26.408366 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:26.408067    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d768ac0-1487-4850-bd9b-c66f73533e97-metrics-certs podName:1d768ac0-1487-4850-bd9b-c66f73533e97 nodeName:}" failed. No retries permitted until 2026-09-15 16:26:28.408050229 +0000 UTC m=+5.037704564 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d768ac0-1487-4850-bd9b-c66f73533e97-metrics-certs") pod "network-metrics-daemon-txzqk" (UID: "1d768ac0-1487-4850-bd9b-c66f73533e97") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 16:26:26.509432 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:26.508885    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d1e544d2-21e5-4abd-82d9-33a82470f13c-original-pull-secret\") pod \"global-pull-secret-syncer-9hbcw\" (UID: \"d1e544d2-21e5-4abd-82d9-33a82470f13c\") " pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:26.509432 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:26.509070    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 16:26:26.509432 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:26.509129    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e544d2-21e5-4abd-82d9-33a82470f13c-original-pull-secret podName:d1e544d2-21e5-4abd-82d9-33a82470f13c nodeName:}" failed. No retries permitted until 2026-09-15 16:26:28.509111642 +0000 UTC m=+5.138765979 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d1e544d2-21e5-4abd-82d9-33a82470f13c-original-pull-secret") pod "global-pull-secret-syncer-9hbcw" (UID: "d1e544d2-21e5-4abd-82d9-33a82470f13c") : object "kube-system"/"original-pull-secret" not registered
Sep 15 16:26:26.609975 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:26.609896    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5h7q\" (UniqueName: \"kubernetes.io/projected/ad430be2-7c5f-4402-a5a7-c8b4359b8c02-kube-api-access-r5h7q\") pod \"network-check-target-sqdn2\" (UID: \"ad430be2-7c5f-4402-a5a7-c8b4359b8c02\") " pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:26:26.610161 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:26.610076    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 16:26:26.610161 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:26.610100    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 16:26:26.610161 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:26.610112    2579 projected.go:196] Error preparing data for projected volume kube-api-access-r5h7q for pod openshift-network-diagnostics/network-check-target-sqdn2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 16:26:26.610324 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:26.610165    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ad430be2-7c5f-4402-a5a7-c8b4359b8c02-kube-api-access-r5h7q podName:ad430be2-7c5f-4402-a5a7-c8b4359b8c02 nodeName:}" failed. No retries permitted until 2026-09-15 16:26:28.610147298 +0000 UTC m=+5.239801641 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-r5h7q" (UniqueName: "kubernetes.io/projected/ad430be2-7c5f-4402-a5a7-c8b4359b8c02-kube-api-access-r5h7q") pod "network-check-target-sqdn2" (UID: "ad430be2-7c5f-4402-a5a7-c8b4359b8c02") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 16:26:26.850574 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:26.850490    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 16:21:24 +0000 UTC" deadline="2028-02-28 07:08:40.847062221 +0000 UTC"
Sep 15 16:26:26.850574 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:26.850527    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12734h42m13.996538923s"
Sep 15 16:26:26.904076 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:26.904045    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:26:26.904215 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:26.904183    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzqk" podUID="1d768ac0-1487-4850-bd9b-c66f73533e97"
Sep 15 16:26:26.904891 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:26.904870    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:26:26.904989 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:26.904971    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sqdn2" podUID="ad430be2-7c5f-4402-a5a7-c8b4359b8c02"
Sep 15 16:26:26.949613 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:26.949513    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-35.ec2.internal" event={"ID":"c3680caa58091ee9b4ffe5030f60f634","Type":"ContainerStarted","Data":"77fd8b7dca53a63eca6aac18a752f09d618cd6e40c6b2253910d217bbbbaf589"}
Sep 15 16:26:27.881851 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:27.881822    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 16:26:27.907124 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:27.907099    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:27.907247 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:27.907195    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hbcw" podUID="d1e544d2-21e5-4abd-82d9-33a82470f13c"
Sep 15 16:26:28.424808 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:28.424760    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d768ac0-1487-4850-bd9b-c66f73533e97-metrics-certs\") pod \"network-metrics-daemon-txzqk\" (UID: \"1d768ac0-1487-4850-bd9b-c66f73533e97\") " pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:26:28.424997 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:28.424892    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 16:26:28.424997 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:28.424949    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d768ac0-1487-4850-bd9b-c66f73533e97-metrics-certs podName:1d768ac0-1487-4850-bd9b-c66f73533e97 nodeName:}" failed. No retries permitted until 2026-09-15 16:26:32.424932835 +0000 UTC m=+9.054587170 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d768ac0-1487-4850-bd9b-c66f73533e97-metrics-certs") pod "network-metrics-daemon-txzqk" (UID: "1d768ac0-1487-4850-bd9b-c66f73533e97") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 16:26:28.526068 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:28.525528    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d1e544d2-21e5-4abd-82d9-33a82470f13c-original-pull-secret\") pod \"global-pull-secret-syncer-9hbcw\" (UID: \"d1e544d2-21e5-4abd-82d9-33a82470f13c\") " pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:28.526068 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:28.525662    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 16:26:28.526068 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:28.525729    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e544d2-21e5-4abd-82d9-33a82470f13c-original-pull-secret podName:d1e544d2-21e5-4abd-82d9-33a82470f13c nodeName:}" failed. No retries permitted until 2026-09-15 16:26:32.52570287 +0000 UTC m=+9.155357209 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d1e544d2-21e5-4abd-82d9-33a82470f13c-original-pull-secret") pod "global-pull-secret-syncer-9hbcw" (UID: "d1e544d2-21e5-4abd-82d9-33a82470f13c") : object "kube-system"/"original-pull-secret" not registered
Sep 15 16:26:28.627398 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:28.626773    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5h7q\" (UniqueName: \"kubernetes.io/projected/ad430be2-7c5f-4402-a5a7-c8b4359b8c02-kube-api-access-r5h7q\") pod \"network-check-target-sqdn2\" (UID: \"ad430be2-7c5f-4402-a5a7-c8b4359b8c02\") " pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:26:28.627398 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:28.626972    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 16:26:28.627398 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:28.626991    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 16:26:28.627398 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:28.627003    2579 projected.go:196] Error preparing data for projected volume kube-api-access-r5h7q for pod openshift-network-diagnostics/network-check-target-sqdn2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 16:26:28.627398 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:28.627056    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ad430be2-7c5f-4402-a5a7-c8b4359b8c02-kube-api-access-r5h7q podName:ad430be2-7c5f-4402-a5a7-c8b4359b8c02 nodeName:}" failed. No retries permitted until 2026-09-15 16:26:32.627039395 +0000 UTC m=+9.256693738 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-r5h7q" (UniqueName: "kubernetes.io/projected/ad430be2-7c5f-4402-a5a7-c8b4359b8c02-kube-api-access-r5h7q") pod "network-check-target-sqdn2" (UID: "ad430be2-7c5f-4402-a5a7-c8b4359b8c02") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 16:26:28.904004 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:28.903936    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:26:28.904337 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:28.904062    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:26:28.904337 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:28.904190    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzqk" podUID="1d768ac0-1487-4850-bd9b-c66f73533e97"
Sep 15 16:26:28.904494 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:28.904459    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sqdn2" podUID="ad430be2-7c5f-4402-a5a7-c8b4359b8c02"
Sep 15 16:26:29.907908 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:29.907408    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:29.907908 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:29.907534    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hbcw" podUID="d1e544d2-21e5-4abd-82d9-33a82470f13c"
Sep 15 16:26:30.903799 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:30.903287    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:26:30.903799 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:30.903390    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzqk" podUID="1d768ac0-1487-4850-bd9b-c66f73533e97"
Sep 15 16:26:30.903799 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:30.903677    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:26:30.903799 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:30.903741    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sqdn2" podUID="ad430be2-7c5f-4402-a5a7-c8b4359b8c02"
Sep 15 16:26:31.903397 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:31.903355    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:31.903825 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:31.903503    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hbcw" podUID="d1e544d2-21e5-4abd-82d9-33a82470f13c"
Sep 15 16:26:32.459258 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:32.459215    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d768ac0-1487-4850-bd9b-c66f73533e97-metrics-certs\") pod \"network-metrics-daemon-txzqk\" (UID: \"1d768ac0-1487-4850-bd9b-c66f73533e97\") " pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:26:32.459429 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:32.459408    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 16:26:32.459489 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:32.459470    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d768ac0-1487-4850-bd9b-c66f73533e97-metrics-certs podName:1d768ac0-1487-4850-bd9b-c66f73533e97 nodeName:}" failed. No retries permitted until 2026-09-15 16:26:40.459449661 +0000 UTC m=+17.089104012 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d768ac0-1487-4850-bd9b-c66f73533e97-metrics-certs") pod "network-metrics-daemon-txzqk" (UID: "1d768ac0-1487-4850-bd9b-c66f73533e97") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 16:26:32.559947 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:32.559910    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d1e544d2-21e5-4abd-82d9-33a82470f13c-original-pull-secret\") pod \"global-pull-secret-syncer-9hbcw\" (UID: \"d1e544d2-21e5-4abd-82d9-33a82470f13c\") " pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:32.560105 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:32.560043    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 16:26:32.560105 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:32.560103    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e544d2-21e5-4abd-82d9-33a82470f13c-original-pull-secret podName:d1e544d2-21e5-4abd-82d9-33a82470f13c nodeName:}" failed. No retries permitted until 2026-09-15 16:26:40.560086428 +0000 UTC m=+17.189740763 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d1e544d2-21e5-4abd-82d9-33a82470f13c-original-pull-secret") pod "global-pull-secret-syncer-9hbcw" (UID: "d1e544d2-21e5-4abd-82d9-33a82470f13c") : object "kube-system"/"original-pull-secret" not registered
Sep 15 16:26:32.660967 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:32.660443    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5h7q\" (UniqueName: \"kubernetes.io/projected/ad430be2-7c5f-4402-a5a7-c8b4359b8c02-kube-api-access-r5h7q\") pod \"network-check-target-sqdn2\" (UID: \"ad430be2-7c5f-4402-a5a7-c8b4359b8c02\") " pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:26:32.660967 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:32.660595    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 16:26:32.660967 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:32.660609    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 16:26:32.660967 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:32.660618    2579 projected.go:196] Error preparing data for projected volume kube-api-access-r5h7q for pod openshift-network-diagnostics/network-check-target-sqdn2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 16:26:32.660967 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:32.660661    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ad430be2-7c5f-4402-a5a7-c8b4359b8c02-kube-api-access-r5h7q podName:ad430be2-7c5f-4402-a5a7-c8b4359b8c02 nodeName:}" failed. No retries permitted until 2026-09-15 16:26:40.660648043 +0000 UTC m=+17.290302379 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-r5h7q" (UniqueName: "kubernetes.io/projected/ad430be2-7c5f-4402-a5a7-c8b4359b8c02-kube-api-access-r5h7q") pod "network-check-target-sqdn2" (UID: "ad430be2-7c5f-4402-a5a7-c8b4359b8c02") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 16:26:32.903747 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:32.903708    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:26:32.904136 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:32.903859    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzqk" podUID="1d768ac0-1487-4850-bd9b-c66f73533e97"
Sep 15 16:26:32.904247 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:32.904215    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:26:32.904328 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:32.904299    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sqdn2" podUID="ad430be2-7c5f-4402-a5a7-c8b4359b8c02"
Sep 15 16:26:33.903696 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:33.903670    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:33.903901 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:33.903769    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hbcw" podUID="d1e544d2-21e5-4abd-82d9-33a82470f13c"
Sep 15 16:26:34.903102 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:34.903073    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:26:34.903261 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:34.903206    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzqk" podUID="1d768ac0-1487-4850-bd9b-c66f73533e97"
Sep 15 16:26:34.903261 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:34.903224    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:26:34.903373 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:34.903315    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sqdn2" podUID="ad430be2-7c5f-4402-a5a7-c8b4359b8c02"
Sep 15 16:26:35.904003 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:35.903974    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:35.904481 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:35.904103    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hbcw" podUID="d1e544d2-21e5-4abd-82d9-33a82470f13c"
Sep 15 16:26:36.903358 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:36.903319    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:26:36.903358 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:36.903365    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:26:36.903594 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:36.903458    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sqdn2" podUID="ad430be2-7c5f-4402-a5a7-c8b4359b8c02"
Sep 15 16:26:36.903648 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:36.903601    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzqk" podUID="1d768ac0-1487-4850-bd9b-c66f73533e97"
Sep 15 16:26:37.903650 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:37.903619    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:37.904062 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:37.903746    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hbcw" podUID="d1e544d2-21e5-4abd-82d9-33a82470f13c"
Sep 15 16:26:38.903222 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:38.903189    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:26:38.903403 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:38.903189    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:26:38.903403 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:38.903285    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sqdn2" podUID="ad430be2-7c5f-4402-a5a7-c8b4359b8c02"
Sep 15 16:26:38.903512 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:38.903411    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzqk" podUID="1d768ac0-1487-4850-bd9b-c66f73533e97"
Sep 15 16:26:39.904056 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:39.904005    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:39.904478 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:39.904185    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hbcw" podUID="d1e544d2-21e5-4abd-82d9-33a82470f13c"
Sep 15 16:26:40.516348 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:40.516317    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d768ac0-1487-4850-bd9b-c66f73533e97-metrics-certs\") pod \"network-metrics-daemon-txzqk\" (UID: \"1d768ac0-1487-4850-bd9b-c66f73533e97\") " pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:26:40.516599 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:40.516483    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 16:26:40.516599 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:40.516559    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d768ac0-1487-4850-bd9b-c66f73533e97-metrics-certs podName:1d768ac0-1487-4850-bd9b-c66f73533e97 nodeName:}" failed. No retries permitted until 2026-09-15 16:26:56.51653737 +0000 UTC m=+33.146191714 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d768ac0-1487-4850-bd9b-c66f73533e97-metrics-certs") pod "network-metrics-daemon-txzqk" (UID: "1d768ac0-1487-4850-bd9b-c66f73533e97") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 16:26:40.617423 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:40.617389    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d1e544d2-21e5-4abd-82d9-33a82470f13c-original-pull-secret\") pod \"global-pull-secret-syncer-9hbcw\" (UID: \"d1e544d2-21e5-4abd-82d9-33a82470f13c\") " pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:40.617588 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:40.617532    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 16:26:40.617588 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:40.617583    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e544d2-21e5-4abd-82d9-33a82470f13c-original-pull-secret podName:d1e544d2-21e5-4abd-82d9-33a82470f13c nodeName:}" failed. No retries permitted until 2026-09-15 16:26:56.617570849 +0000 UTC m=+33.247225179 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d1e544d2-21e5-4abd-82d9-33a82470f13c-original-pull-secret") pod "global-pull-secret-syncer-9hbcw" (UID: "d1e544d2-21e5-4abd-82d9-33a82470f13c") : object "kube-system"/"original-pull-secret" not registered
Sep 15 16:26:40.717905 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:40.717875    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5h7q\" (UniqueName: \"kubernetes.io/projected/ad430be2-7c5f-4402-a5a7-c8b4359b8c02-kube-api-access-r5h7q\") pod \"network-check-target-sqdn2\" (UID: \"ad430be2-7c5f-4402-a5a7-c8b4359b8c02\") " pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:26:40.718079 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:40.718055    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 16:26:40.718152 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:40.718085    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 16:26:40.718152 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:40.718097    2579 projected.go:196] Error preparing data for projected volume kube-api-access-r5h7q for pod openshift-network-diagnostics/network-check-target-sqdn2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 16:26:40.718283 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:40.718153    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ad430be2-7c5f-4402-a5a7-c8b4359b8c02-kube-api-access-r5h7q podName:ad430be2-7c5f-4402-a5a7-c8b4359b8c02 nodeName:}" failed. No retries permitted until 2026-09-15 16:26:56.718136517 +0000 UTC m=+33.347790861 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-r5h7q" (UniqueName: "kubernetes.io/projected/ad430be2-7c5f-4402-a5a7-c8b4359b8c02-kube-api-access-r5h7q") pod "network-check-target-sqdn2" (UID: "ad430be2-7c5f-4402-a5a7-c8b4359b8c02") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 16:26:40.903464 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:40.903390    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:26:40.903464 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:40.903420    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:26:40.903647 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:40.903513    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzqk" podUID="1d768ac0-1487-4850-bd9b-c66f73533e97"
Sep 15 16:26:40.903689 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:40.903652    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sqdn2" podUID="ad430be2-7c5f-4402-a5a7-c8b4359b8c02"
Sep 15 16:26:41.903797 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:41.903756    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:41.904207 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:41.903898    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hbcw" podUID="d1e544d2-21e5-4abd-82d9-33a82470f13c"
Sep 15 16:26:42.904086 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:42.904059    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:26:42.904522 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:42.904187    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzqk" podUID="1d768ac0-1487-4850-bd9b-c66f73533e97"
Sep 15 16:26:42.904522 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:42.904225    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:26:42.904522 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:42.904341    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sqdn2" podUID="ad430be2-7c5f-4402-a5a7-c8b4359b8c02"
Sep 15 16:26:43.904600 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:43.904468    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:43.904970 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:43.904662    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hbcw" podUID="d1e544d2-21e5-4abd-82d9-33a82470f13c"
Sep 15 16:26:43.983601 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:43.983577    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zr9w5" event={"ID":"7a83b81c-a949-41e4-9e47-e7305b24d4a7","Type":"ContainerStarted","Data":"05d7ebfdcec0e0dc1d39dc45dcb8377e573c2a5867144dbf819e781a1b622295"}
Sep 15 16:26:43.985656 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:43.985633    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn" event={"ID":"44ca88f0-c149-443e-a2d1-a30964775818","Type":"ContainerStarted","Data":"47894dbc289e107faeeb56ed7aa92cd89d4bbfb0a8eceededc2efcfca363b3e0"}
Sep 15 16:26:43.985761 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:43.985700    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn" event={"ID":"44ca88f0-c149-443e-a2d1-a30964775818","Type":"ContainerStarted","Data":"acc082dde4bcdebfe858b15ba21c5f9bbf8532d2630ef857690d52a55e7afb26"}
Sep 15 16:26:43.985761 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:43.985743    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn" event={"ID":"44ca88f0-c149-443e-a2d1-a30964775818","Type":"ContainerStarted","Data":"095320e0f2d1b39836414e5f337751527a59b1177efedf991f02b37659c2f692"}
Sep 15 16:26:43.987408 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:43.987120    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-35.ec2.internal" event={"ID":"269bca4735829fdfafb55c4d16c2e698","Type":"ContainerStarted","Data":"b28d56655278a6f164f8df68ae43e568998c8dd79cb77fb444b3b1a60d8ac660"}
Sep 15 16:26:44.026962 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:44.026915    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zr9w5" podStartSLOduration=2.719205132 podStartE2EDuration="21.026897328s" totalImagesPullingTime="18.307692196s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 16:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:26:44.0141295 +0000 UTC m=+20.643783852" watchObservedRunningTime="2026-09-15 16:26:44.026897328 +0000 UTC m=+20.656551679"
Sep 15 16:26:44.904201 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:44.904031    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:26:44.904276 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:44.904036    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:26:44.904333 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:44.904238    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sqdn2" podUID="ad430be2-7c5f-4402-a5a7-c8b4359b8c02"
Sep 15 16:26:44.904376 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:44.904318    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzqk" podUID="1d768ac0-1487-4850-bd9b-c66f73533e97"
Sep 15 16:26:44.990349 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:44.990322    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8zk2f" event={"ID":"f89021a6-bc8a-4f31-a76c-87c39f332c20","Type":"ContainerStarted","Data":"05f853d81c162c18920305c93776643d10d042564dcab933753d5ff992bd75db"}
Sep 15 16:26:44.992502 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:44.992477    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cs7tq" event={"ID":"bf4e438a-fbc7-401f-b97c-bbb7bc4a0495","Type":"ContainerStarted","Data":"37ae8d0a55d2d9e0603c7cd0b465e747ccd9913dcb141a36ef3b2ac23525488c"}
Sep 15 16:26:44.995199 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:44.995176    2579 generic.go:356] "Generic (PLEG): container finished" podID="b17d9664-4ce9-449a-8826-af717fd47f97" containerID="1b3cde77b307ca279effee125b0f5cec73f59d45e970757a7a04567a2e988776" exitCode=0
Sep 15 16:26:44.995462 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:44.995291    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cvgk4" event={"ID":"b17d9664-4ce9-449a-8826-af717fd47f97","Type":"ContainerDied","Data":"1b3cde77b307ca279effee125b0f5cec73f59d45e970757a7a04567a2e988776"}
Sep 15 16:26:44.997972 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:44.997909    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-s7d5t" event={"ID":"4893f9dd-7ff1-419a-9e27-c12efdb8285d","Type":"ContainerStarted","Data":"4236655b0f3b65767e414e1f96ac086b8fadff59555fd9230eaa0fb2b7215e3a"}
Sep 15 16:26:45.001538 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:45.001521    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4rdn_44ca88f0-c149-443e-a2d1-a30964775818/ovn-acl-logging/0.log"
Sep 15 16:26:45.001979 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:45.001953    2579 generic.go:356] "Generic (PLEG): container finished" podID="44ca88f0-c149-443e-a2d1-a30964775818" containerID="acc082dde4bcdebfe858b15ba21c5f9bbf8532d2630ef857690d52a55e7afb26" exitCode=1
Sep 15 16:26:45.002066 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:45.002026    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn" event={"ID":"44ca88f0-c149-443e-a2d1-a30964775818","Type":"ContainerDied","Data":"acc082dde4bcdebfe858b15ba21c5f9bbf8532d2630ef857690d52a55e7afb26"}
Sep 15 16:26:45.002119 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:45.002092    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn" event={"ID":"44ca88f0-c149-443e-a2d1-a30964775818","Type":"ContainerStarted","Data":"9d5060e47fd11bf5a8dbeb059b46864bf4940d4d077544463b866abae24733ba"}
Sep 15 16:26:45.002158 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:45.002123    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn" event={"ID":"44ca88f0-c149-443e-a2d1-a30964775818","Type":"ContainerStarted","Data":"2acc3c6f572d7fb254281b77766d5340957b71828ecde793a307398c8a5042e6"}
Sep 15 16:26:45.002219 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:45.002158    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn" event={"ID":"44ca88f0-c149-443e-a2d1-a30964775818","Type":"ContainerStarted","Data":"1a7ae5b7aa48c6dec378a7b25e833d2bec5c604ebb8514c6162ecee5a9d46f49"}
Sep 15 16:26:45.003565 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:45.003544    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rw6mp" event={"ID":"3c332c22-48cd-4420-9a35-e40f98ec4344","Type":"ContainerStarted","Data":"1f3bbc8f7ba20e8e7cfc50c890cb4d26698b47d7ce74a6154eed833a8035e709"}
Sep 15 16:26:45.005115 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:45.005095    2579 generic.go:356] "Generic (PLEG): container finished" podID="c3680caa58091ee9b4ffe5030f60f634" containerID="5b59fd459c4061dde51c369526e62fd98c8bae79d735bd57be2af3f01160536d" exitCode=0
Sep 15 16:26:45.005196 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:45.005179    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-35.ec2.internal" event={"ID":"c3680caa58091ee9b4ffe5030f60f634","Type":"ContainerDied","Data":"5b59fd459c4061dde51c369526e62fd98c8bae79d735bd57be2af3f01160536d"}
Sep 15 16:26:45.005667 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:45.005539    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-35.ec2.internal" podStartSLOduration=21.005525518 podStartE2EDuration="21.005525518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 16:26:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:26:44.026973934 +0000 UTC m=+20.656628286" watchObservedRunningTime="2026-09-15 16:26:45.005525518 +0000 UTC m=+21.635179872"
Sep 15 16:26:45.005751 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:45.005712    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-8zk2f" podStartSLOduration=3.950375965 podStartE2EDuration="22.005702679s" totalImagesPullingTime="18.055326714s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 16:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:26:45.00535789 +0000 UTC m=+21.635012241" watchObservedRunningTime="2026-09-15 16:26:45.005702679 +0000 UTC m=+21.635357033"
Sep 15 16:26:45.006953 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:45.006931    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dqm7s" event={"ID":"ba7071d8-d459-4c22-ae2f-1ced3f8871ce","Type":"ContainerStarted","Data":"6a648c1c79ddc613cee39a898ad79afc21abf0f5257d2d8fc4d7a04b0e6e578b"}
Sep 15 16:26:45.009053 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:45.008867    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-btdbz" event={"ID":"fdb8314d-7223-4926-b551-8bb1701052fd","Type":"ContainerStarted","Data":"6698edbdc8254c391280d53e1da648406349a5617945e847c433b0da0e4a8453"}
Sep 15 16:26:45.028295 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:45.028077    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-s7d5t" podStartSLOduration=3.653795465 podStartE2EDuration="22.028060761s" totalImagesPullingTime="18.374265296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 16:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:26:45.027461917 +0000 UTC m=+21.657116269" watchObservedRunningTime="2026-09-15 16:26:45.028060761 +0000 UTC m=+21.657715114"
Sep 15 16:26:45.129846 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:45.129774    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-cs7tq" podStartSLOduration=3.721189133 podStartE2EDuration="22.129757076s" totalImagesPullingTime="18.408567943s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 16:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:26:45.109015082 +0000 UTC m=+21.738669436" watchObservedRunningTime="2026-09-15 16:26:45.129757076 +0000 UTC m=+21.759411438"
Sep 15 16:26:45.130340 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:45.130300    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-btdbz" podStartSLOduration=7.91565445 podStartE2EDuration="22.130289231s" totalImagesPullingTime="14.214634781s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 16:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:26:45.130239325 +0000 UTC m=+21.759893677" watchObservedRunningTime="2026-09-15 16:26:45.130289231 +0000 UTC m=+21.759943585"
Sep 15 16:26:45.158082 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:45.157993    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-dqm7s" podStartSLOduration=3.392402946 podStartE2EDuration="22.157975299s" totalImagesPullingTime="18.765572353s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 16:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:26:45.1572711 +0000 UTC m=+21.786925452" watchObservedRunningTime="2026-09-15 16:26:45.157975299 +0000 UTC m=+21.787629652"
Sep 15 16:26:45.426319 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:45.426289    2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 15 16:26:45.858549 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:45.858462    2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-15T16:26:45.42631972Z","UUID":"b0208af4-c562-47dc-988c-1378225dba1d","Handler":null,"Name":"","Endpoint":""}
Sep 15 16:26:45.860283 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:45.860263    2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 15 16:26:45.860413 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:45.860295    2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 15 16:26:45.904128 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:45.904104    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:45.904294 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:45.904222    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hbcw" podUID="d1e544d2-21e5-4abd-82d9-33a82470f13c"
Sep 15 16:26:46.011994 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:46.011971    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rw6mp" event={"ID":"3c332c22-48cd-4420-9a35-e40f98ec4344","Type":"ContainerStarted","Data":"69f687a03a75fcbf3fd61ac772b9f46a67581a4aa614654d80863238de88aca0"}
Sep 15 16:26:46.013716 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:46.013694    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-35.ec2.internal" event={"ID":"c3680caa58091ee9b4ffe5030f60f634","Type":"ContainerStarted","Data":"5e112e5439fefe205dbd0242214b4e53044b55e7e301c35557759ffbfc40b837"}
Sep 15 16:26:46.032657 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:46.032607    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-35.ec2.internal" podStartSLOduration=22.032590991 podStartE2EDuration="22.032590991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 16:26:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:26:46.031834123 +0000 UTC m=+22.661488475" watchObservedRunningTime="2026-09-15 16:26:46.032590991 +0000 UTC m=+22.662245343"
Sep 15 16:26:46.231359 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:46.231327    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-s7d5t"
Sep 15 16:26:46.232227 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:46.232201    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-s7d5t"
Sep 15 16:26:46.904011 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:46.903925    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:26:46.904011 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:46.903959    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:26:46.904220 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:46.904054    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzqk" podUID="1d768ac0-1487-4850-bd9b-c66f73533e97"
Sep 15 16:26:46.904278 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:46.904221    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sqdn2" podUID="ad430be2-7c5f-4402-a5a7-c8b4359b8c02"
Sep 15 16:26:47.019339 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:47.019318    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4rdn_44ca88f0-c149-443e-a2d1-a30964775818/ovn-acl-logging/0.log"
Sep 15 16:26:47.019814 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:47.019755    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn" event={"ID":"44ca88f0-c149-443e-a2d1-a30964775818","Type":"ContainerStarted","Data":"e16956fe5cd323e2990fc1660653f9eff23d3df5b0e4638e1f55729a22c6ba04"}
Sep 15 16:26:47.022098 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:47.022010    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rw6mp" event={"ID":"3c332c22-48cd-4420-9a35-e40f98ec4344","Type":"ContainerStarted","Data":"a923c7e340bd04c50cbeb4f03e1fb28b49389022ae96c44eecd8388d6d922d36"}
Sep 15 16:26:47.022449 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:47.022424    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-s7d5t"
Sep 15 16:26:47.022881 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:47.022864    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-s7d5t"
Sep 15 16:26:47.047253 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:47.047208    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rw6mp" podStartSLOduration=3.8674015710000003 podStartE2EDuration="24.047193546s" totalImagesPullingTime="20.179791975s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 16:26:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:26:47.047181014 +0000 UTC m=+23.676835367" watchObservedRunningTime="2026-09-15 16:26:47.047193546 +0000 UTC m=+23.676847893"
Sep 15 16:26:47.903257 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:47.903232    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:47.903427 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:47.903344    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hbcw" podUID="d1e544d2-21e5-4abd-82d9-33a82470f13c"
Sep 15 16:26:48.903961 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:48.903892    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:26:48.904348 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:48.904028    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sqdn2" podUID="ad430be2-7c5f-4402-a5a7-c8b4359b8c02"
Sep 15 16:26:48.904348 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:48.904073    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:26:48.904348 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:48.904178    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzqk" podUID="1d768ac0-1487-4850-bd9b-c66f73533e97"
Sep 15 16:26:49.903622 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:49.903473    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:49.903751 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:49.903715    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hbcw" podUID="d1e544d2-21e5-4abd-82d9-33a82470f13c"
Sep 15 16:26:50.029582 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:50.029556    2579 generic.go:356] "Generic (PLEG): container finished" podID="b17d9664-4ce9-449a-8826-af717fd47f97" containerID="a4937c8c64fc8454d03c3b6c214309d1fbdcdf4bc80397d1a12a0fcbb27aa654" exitCode=0
Sep 15 16:26:50.030165 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:50.029634    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cvgk4" event={"ID":"b17d9664-4ce9-449a-8826-af717fd47f97","Type":"ContainerDied","Data":"a4937c8c64fc8454d03c3b6c214309d1fbdcdf4bc80397d1a12a0fcbb27aa654"}
Sep 15 16:26:50.033180 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:50.033164    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4rdn_44ca88f0-c149-443e-a2d1-a30964775818/ovn-acl-logging/0.log"
Sep 15 16:26:50.033553 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:50.033536    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn" event={"ID":"44ca88f0-c149-443e-a2d1-a30964775818","Type":"ContainerStarted","Data":"3455ee0863fbc64e658f3a2c0fcef2a4427705cfe0581142c4b0eefb43a975fe"}
Sep 15 16:26:50.033763 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:50.033739    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:50.033862 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:50.033829    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:50.033913 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:50.033865    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:50.033913 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:50.033891    2579 scope.go:117] "RemoveContainer" containerID="acc082dde4bcdebfe858b15ba21c5f9bbf8532d2630ef857690d52a55e7afb26"
Sep 15 16:26:50.049279 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:50.049263    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:50.049635 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:50.049620    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:26:50.903465 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:50.903442    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:26:50.903547 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:50.903535    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sqdn2" podUID="ad430be2-7c5f-4402-a5a7-c8b4359b8c02"
Sep 15 16:26:50.903603 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:50.903587    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:26:50.903710 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:50.903688    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzqk" podUID="1d768ac0-1487-4850-bd9b-c66f73533e97"
Sep 15 16:26:51.040638 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:51.040578    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4rdn_44ca88f0-c149-443e-a2d1-a30964775818/ovn-acl-logging/0.log"
Sep 15 16:26:51.041155 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:51.041067    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn" event={"ID":"44ca88f0-c149-443e-a2d1-a30964775818","Type":"ContainerStarted","Data":"b833c8e33dfae031ad39be0a338850ab79c70eb635b36886c98a47db642975e3"}
Sep 15 16:26:51.043234 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:51.043209    2579 generic.go:356] "Generic (PLEG): container finished" podID="b17d9664-4ce9-449a-8826-af717fd47f97" containerID="6a38624c69669fab6f1136c696ea8242daf69c9e072d0090341b8219248c2666" exitCode=0
Sep 15 16:26:51.043323 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:51.043240    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cvgk4" event={"ID":"b17d9664-4ce9-449a-8826-af717fd47f97","Type":"ContainerDied","Data":"6a38624c69669fab6f1136c696ea8242daf69c9e072d0090341b8219248c2666"}
Sep 15 16:26:51.090468 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:51.090425    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn" podStartSLOduration=9.697093207 podStartE2EDuration="28.09041098s" totalImagesPullingTime="18.393317773s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 16:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:26:51.090008578 +0000 UTC m=+27.719662928" watchObservedRunningTime="2026-09-15 16:26:51.09041098 +0000 UTC m=+27.720065332"
Sep 15 16:26:51.327739 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:51.327566    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-txzqk"]
Sep 15 16:26:51.327849 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:51.327753    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:26:51.327886 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:51.327847    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzqk" podUID="1d768ac0-1487-4850-bd9b-c66f73533e97"
Sep 15 16:26:51.330827 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:51.330800    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9hbcw"]
Sep 15 16:26:51.330937 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:51.330913    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:51.331037 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:51.331015    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hbcw" podUID="d1e544d2-21e5-4abd-82d9-33a82470f13c"
Sep 15 16:26:51.331515 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:51.331497    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sqdn2"]
Sep 15 16:26:51.331590 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:51.331579    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:26:51.331688 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:51.331668    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sqdn2" podUID="ad430be2-7c5f-4402-a5a7-c8b4359b8c02"
Sep 15 16:26:52.047774 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:52.047748    2579 generic.go:356] "Generic (PLEG): container finished" podID="b17d9664-4ce9-449a-8826-af717fd47f97" containerID="3d6be578f7deccc9d3f25e24fc36fcf01cae324bddd71f32f1da3ce5d3e51b75" exitCode=0
Sep 15 16:26:52.048033 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:52.047836    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cvgk4" event={"ID":"b17d9664-4ce9-449a-8826-af717fd47f97","Type":"ContainerDied","Data":"3d6be578f7deccc9d3f25e24fc36fcf01cae324bddd71f32f1da3ce5d3e51b75"}
Sep 15 16:26:52.903773 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:52.903712    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:26:52.903897 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:52.903831    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:26:52.903950 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:52.903834    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzqk" podUID="1d768ac0-1487-4850-bd9b-c66f73533e97"
Sep 15 16:26:52.903950 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:52.903920    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sqdn2" podUID="ad430be2-7c5f-4402-a5a7-c8b4359b8c02"
Sep 15 16:26:52.904059 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:52.903958    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:52.904103 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:52.904057    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hbcw" podUID="d1e544d2-21e5-4abd-82d9-33a82470f13c"
Sep 15 16:26:54.904120 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:54.904043    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:54.904120 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:54.904074    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:26:54.904529 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:54.904147    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hbcw" podUID="d1e544d2-21e5-4abd-82d9-33a82470f13c"
Sep 15 16:26:54.904529 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:54.904188    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:26:54.904529 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:54.904286    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzqk" podUID="1d768ac0-1487-4850-bd9b-c66f73533e97"
Sep 15 16:26:54.904529 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:54.904359    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sqdn2" podUID="ad430be2-7c5f-4402-a5a7-c8b4359b8c02"
Sep 15 16:26:56.266035 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.266006    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-btdbz_fdb8314d-7223-4926-b551-8bb1701052fd/dns-node-resolver/0.log"
Sep 15 16:26:56.537021 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.536959    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d768ac0-1487-4850-bd9b-c66f73533e97-metrics-certs\") pod \"network-metrics-daemon-txzqk\" (UID: \"1d768ac0-1487-4850-bd9b-c66f73533e97\") " pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:26:56.537145 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:56.537079    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 16:26:56.537145 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:56.537132    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d768ac0-1487-4850-bd9b-c66f73533e97-metrics-certs podName:1d768ac0-1487-4850-bd9b-c66f73533e97 nodeName:}" failed. No retries permitted until 2026-09-15 16:27:28.537113659 +0000 UTC m=+65.166767989 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d768ac0-1487-4850-bd9b-c66f73533e97-metrics-certs") pod "network-metrics-daemon-txzqk" (UID: "1d768ac0-1487-4850-bd9b-c66f73533e97") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 16:26:56.637727 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.637702    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d1e544d2-21e5-4abd-82d9-33a82470f13c-original-pull-secret\") pod \"global-pull-secret-syncer-9hbcw\" (UID: \"d1e544d2-21e5-4abd-82d9-33a82470f13c\") " pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:56.637898 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:56.637878    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 16:26:56.637951 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:56.637939    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e544d2-21e5-4abd-82d9-33a82470f13c-original-pull-secret podName:d1e544d2-21e5-4abd-82d9-33a82470f13c nodeName:}" failed. No retries permitted until 2026-09-15 16:27:28.637924547 +0000 UTC m=+65.267578881 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d1e544d2-21e5-4abd-82d9-33a82470f13c-original-pull-secret") pod "global-pull-secret-syncer-9hbcw" (UID: "d1e544d2-21e5-4abd-82d9-33a82470f13c") : object "kube-system"/"original-pull-secret" not registered
Sep 15 16:26:56.713673 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.713645    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-35.ec2.internal" event="NodeReady"
Sep 15 16:26:56.713836 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.713800    2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 15 16:26:56.738432 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.738404    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5h7q\" (UniqueName: \"kubernetes.io/projected/ad430be2-7c5f-4402-a5a7-c8b4359b8c02-kube-api-access-r5h7q\") pod \"network-check-target-sqdn2\" (UID: \"ad430be2-7c5f-4402-a5a7-c8b4359b8c02\") " pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:26:56.738572 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:56.738523    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 16:26:56.738572 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:56.738537    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 16:26:56.738572 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:56.738545    2579 projected.go:196] Error preparing data for projected volume kube-api-access-r5h7q for pod openshift-network-diagnostics/network-check-target-sqdn2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 16:26:56.738733 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:56.738585    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ad430be2-7c5f-4402-a5a7-c8b4359b8c02-kube-api-access-r5h7q podName:ad430be2-7c5f-4402-a5a7-c8b4359b8c02 nodeName:}" failed. No retries permitted until 2026-09-15 16:27:28.738574238 +0000 UTC m=+65.368228581 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-r5h7q" (UniqueName: "kubernetes.io/projected/ad430be2-7c5f-4402-a5a7-c8b4359b8c02-kube-api-access-r5h7q") pod "network-check-target-sqdn2" (UID: "ad430be2-7c5f-4402-a5a7-c8b4359b8c02") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 16:26:56.757352 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.757330    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8c846c645-h9j62"]
Sep 15 16:26:56.783433 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.783411    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-27hdl"]
Sep 15 16:26:56.784076 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.784052    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:26:56.787618 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.787570    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 15 16:26:56.787834 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.787818    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 15 16:26:56.788049 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.788033    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 15 16:26:56.788252 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.788232    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5vwhx\""
Sep 15 16:26:56.794661 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.794614    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 15 16:26:56.797391 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.797346    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8c846c645-h9j62"]
Sep 15 16:26:56.797391 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.797385    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-45s5g"]
Sep 15 16:26:56.797528 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.797507    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-27hdl"
Sep 15 16:26:56.800173 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.800112    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 15 16:26:56.800173 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.800118    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 15 16:26:56.800173 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.800149    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 15 16:26:56.800357 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.800149    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 15 16:26:56.800357 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.800216    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zvqwt\""
Sep 15 16:26:56.817610 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.817591    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-27hdl"]
Sep 15 16:26:56.817702 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.817637    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2kxsk"]
Sep 15 16:26:56.817767 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.817753    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-45s5g"
Sep 15 16:26:56.820325 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.820151    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 15 16:26:56.820325 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.820181    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-m9qf9\""
Sep 15 16:26:56.820476 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.820454    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 15 16:26:56.829597 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.829577    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-45s5g"]
Sep 15 16:26:56.829703 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.829612    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2kxsk"]
Sep 15 16:26:56.829703 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.829693    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2kxsk"
Sep 15 16:26:56.832215 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.832197    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 15 16:26:56.833133 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.833118    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rwdks\""
Sep 15 16:26:56.833208 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.833118    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 15 16:26:56.833503 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.833485    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 15 16:26:56.903383 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.903358    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:26:56.903510 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.903386    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:26:56.903889 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.903870    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:26:56.908218 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.908200    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 15 16:26:56.908218 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.908200    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-v6gf7\""
Sep 15 16:26:56.908349 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.908233    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 15 16:26:56.908575 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.908521    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 15 16:26:56.908575 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.908566    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 15 16:26:56.908705 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.908598    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zqb5b\""
Sep 15 16:26:56.939457 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.939429    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-27hdl\" (UID: \"abc7e760-ef1c-48bd-8d65-c5ad19f3f422\") " pod="openshift-insights/insights-runtime-extractor-27hdl"
Sep 15 16:26:56.939543 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.939471    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65szf\" (UniqueName: \"kubernetes.io/projected/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-kube-api-access-65szf\") pod \"insights-runtime-extractor-27hdl\" (UID: \"abc7e760-ef1c-48bd-8d65-c5ad19f3f422\") " pod="openshift-insights/insights-runtime-extractor-27hdl"
Sep 15 16:26:56.939543 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.939497    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-registry-tls\") pod \"image-registry-8c846c645-h9j62\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") " pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:26:56.939670 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.939555    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-trusted-ca\") pod \"image-registry-8c846c645-h9j62\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") " pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:26:56.939670 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.939595    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-27hdl\" (UID: \"abc7e760-ef1c-48bd-8d65-c5ad19f3f422\") " pod="openshift-insights/insights-runtime-extractor-27hdl"
Sep 15 16:26:56.939670 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.939621    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct8fd\" (UniqueName: \"kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-kube-api-access-ct8fd\") pod \"image-registry-8c846c645-h9j62\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") " pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:26:56.939670 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.939653    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-ca-trust-extracted\") pod \"image-registry-8c846c645-h9j62\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") " pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:26:56.939869 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.939700    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-installation-pull-secrets\") pod \"image-registry-8c846c645-h9j62\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") " pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:26:56.939869 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.939745    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4r6b\" (UniqueName: \"kubernetes.io/projected/8d780ae3-adfc-447b-9605-30793d9fc5c1-kube-api-access-x4r6b\") pod \"dns-default-45s5g\" (UID: \"8d780ae3-adfc-447b-9605-30793d9fc5c1\") " pod="openshift-dns/dns-default-45s5g"
Sep 15 16:26:56.939869 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.939819    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-data-volume\") pod \"insights-runtime-extractor-27hdl\" (UID: \"abc7e760-ef1c-48bd-8d65-c5ad19f3f422\") " pod="openshift-insights/insights-runtime-extractor-27hdl"
Sep 15 16:26:56.939869 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.939864    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/84da0f72-32d0-4f59-9cad-f2c59cc71a0b-cert\") pod \"ingress-canary-2kxsk\" (UID: \"84da0f72-32d0-4f59-9cad-f2c59cc71a0b\") " pod="openshift-ingress-canary/ingress-canary-2kxsk"
Sep 15 16:26:56.940051 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.939891    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgh7d\" (UniqueName: \"kubernetes.io/projected/84da0f72-32d0-4f59-9cad-f2c59cc71a0b-kube-api-access-rgh7d\") pod \"ingress-canary-2kxsk\" (UID: \"84da0f72-32d0-4f59-9cad-f2c59cc71a0b\") " pod="openshift-ingress-canary/ingress-canary-2kxsk"
Sep 15 16:26:56.940051 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.939918    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-image-registry-private-configuration\") pod \"image-registry-8c846c645-h9j62\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") " pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:26:56.940051 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.939942    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-registry-certificates\") pod \"image-registry-8c846c645-h9j62\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") " pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:26:56.940051 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.939965    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8d780ae3-adfc-447b-9605-30793d9fc5c1-config-volume\") pod \"dns-default-45s5g\" (UID: \"8d780ae3-adfc-447b-9605-30793d9fc5c1\") " pod="openshift-dns/dns-default-45s5g"
Sep 15 16:26:56.940051 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.939986    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d780ae3-adfc-447b-9605-30793d9fc5c1-tmp-dir\") pod \"dns-default-45s5g\" (UID: \"8d780ae3-adfc-447b-9605-30793d9fc5c1\") " pod="openshift-dns/dns-default-45s5g"
Sep 15 16:26:56.940211 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.940044    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-crio-socket\") pod \"insights-runtime-extractor-27hdl\" (UID: \"abc7e760-ef1c-48bd-8d65-c5ad19f3f422\") " pod="openshift-insights/insights-runtime-extractor-27hdl"
Sep 15 16:26:56.940211 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.940122    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-bound-sa-token\") pod \"image-registry-8c846c645-h9j62\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") " pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:26:56.940211 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:56.940142    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8d780ae3-adfc-447b-9605-30793d9fc5c1-metrics-tls\") pod \"dns-default-45s5g\" (UID: \"8d780ae3-adfc-447b-9605-30793d9fc5c1\") " pod="openshift-dns/dns-default-45s5g"
Sep 15 16:26:57.040811 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.040730    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-ca-trust-extracted\") pod \"image-registry-8c846c645-h9j62\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") " pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:26:57.040811 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.040799    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-installation-pull-secrets\") pod \"image-registry-8c846c645-h9j62\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") " pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:26:57.040976 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.040828    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4r6b\" (UniqueName: \"kubernetes.io/projected/8d780ae3-adfc-447b-9605-30793d9fc5c1-kube-api-access-x4r6b\") pod \"dns-default-45s5g\" (UID: \"8d780ae3-adfc-447b-9605-30793d9fc5c1\") " pod="openshift-dns/dns-default-45s5g"
Sep 15 16:26:57.040976 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.040863    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-data-volume\") pod \"insights-runtime-extractor-27hdl\" (UID: \"abc7e760-ef1c-48bd-8d65-c5ad19f3f422\") " pod="openshift-insights/insights-runtime-extractor-27hdl"
Sep 15 16:26:57.040976 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.040908    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/84da0f72-32d0-4f59-9cad-f2c59cc71a0b-cert\") pod \"ingress-canary-2kxsk\" (UID: \"84da0f72-32d0-4f59-9cad-f2c59cc71a0b\") " pod="openshift-ingress-canary/ingress-canary-2kxsk"
Sep 15 16:26:57.040976 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.040937    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgh7d\" (UniqueName: \"kubernetes.io/projected/84da0f72-32d0-4f59-9cad-f2c59cc71a0b-kube-api-access-rgh7d\") pod \"ingress-canary-2kxsk\" (UID: \"84da0f72-32d0-4f59-9cad-f2c59cc71a0b\") " pod="openshift-ingress-canary/ingress-canary-2kxsk"
Sep 15 16:26:57.040976 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.040964    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-image-registry-private-configuration\") pod \"image-registry-8c846c645-h9j62\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") " pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:26:57.041156 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.040994    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-registry-certificates\") pod \"image-registry-8c846c645-h9j62\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") " pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:26:57.041156 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.041016    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8d780ae3-adfc-447b-9605-30793d9fc5c1-config-volume\") pod \"dns-default-45s5g\" (UID: \"8d780ae3-adfc-447b-9605-30793d9fc5c1\") " pod="openshift-dns/dns-default-45s5g"
Sep 15 16:26:57.041156 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.041040    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d780ae3-adfc-447b-9605-30793d9fc5c1-tmp-dir\") pod \"dns-default-45s5g\" (UID: \"8d780ae3-adfc-447b-9605-30793d9fc5c1\") " pod="openshift-dns/dns-default-45s5g"
Sep 15 16:26:57.041156 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.041078    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-crio-socket\") pod \"insights-runtime-extractor-27hdl\" (UID: \"abc7e760-ef1c-48bd-8d65-c5ad19f3f422\") " pod="openshift-insights/insights-runtime-extractor-27hdl"
Sep 15 16:26:57.041156 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.041123    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-bound-sa-token\") pod \"image-registry-8c846c645-h9j62\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") " pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:26:57.041156 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.041138    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-ca-trust-extracted\") pod \"image-registry-8c846c645-h9j62\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") " pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:26:57.041156 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.041153    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8d780ae3-adfc-447b-9605-30793d9fc5c1-metrics-tls\") pod \"dns-default-45s5g\" (UID: \"8d780ae3-adfc-447b-9605-30793d9fc5c1\") " pod="openshift-dns/dns-default-45s5g"
Sep 15 16:26:57.041431 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.041196    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-27hdl\" (UID: \"abc7e760-ef1c-48bd-8d65-c5ad19f3f422\") " pod="openshift-insights/insights-runtime-extractor-27hdl"
Sep 15 16:26:57.041431 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.041222    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65szf\" (UniqueName: \"kubernetes.io/projected/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-kube-api-access-65szf\") pod \"insights-runtime-extractor-27hdl\" (UID: \"abc7e760-ef1c-48bd-8d65-c5ad19f3f422\") " pod="openshift-insights/insights-runtime-extractor-27hdl"
Sep 15 16:26:57.041431 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:57.041246    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 16:26:57.041431 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.041253    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-registry-tls\") pod \"image-registry-8c846c645-h9j62\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") " pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:26:57.041431 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.041276    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-trusted-ca\") pod \"image-registry-8c846c645-h9j62\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") " pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:26:57.041431 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:57.041301    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d780ae3-adfc-447b-9605-30793d9fc5c1-metrics-tls podName:8d780ae3-adfc-447b-9605-30793d9fc5c1 nodeName:}" failed. No retries permitted until 2026-09-15 16:26:57.541282193 +0000 UTC m=+34.170936526 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8d780ae3-adfc-447b-9605-30793d9fc5c1-metrics-tls") pod "dns-default-45s5g" (UID: "8d780ae3-adfc-447b-9605-30793d9fc5c1") : secret "dns-default-metrics-tls" not found
Sep 15 16:26:57.041431 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.041330    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-27hdl\" (UID: \"abc7e760-ef1c-48bd-8d65-c5ad19f3f422\") " pod="openshift-insights/insights-runtime-extractor-27hdl"
Sep 15 16:26:57.041431 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.041362    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct8fd\" (UniqueName: \"kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-kube-api-access-ct8fd\") pod \"image-registry-8c846c645-h9j62\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") " pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:26:57.041971 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:57.041946    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 16:26:57.042064 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:57.042008    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-kube-rbac-proxy-cm podName:abc7e760-ef1c-48bd-8d65-c5ad19f3f422 nodeName:}" failed. No retries permitted until 2026-09-15 16:26:57.541991434 +0000 UTC m=+34.171645765 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-kube-rbac-proxy-cm") pod "insights-runtime-extractor-27hdl" (UID: "abc7e760-ef1c-48bd-8d65-c5ad19f3f422") : configmap "kube-rbac-proxy" not found
Sep 15 16:26:57.042221 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.042197    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-trusted-ca\") pod \"image-registry-8c846c645-h9j62\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") " pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:26:57.042314 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:57.042290    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 16:26:57.042314 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:57.042302    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8c846c645-h9j62: secret "image-registry-tls" not found
Sep 15 16:26:57.042398 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:57.042339    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-registry-tls podName:563c6d5b-c270-41de-bed4-b2aeb8fe8f42 nodeName:}" failed. No retries permitted until 2026-09-15 16:26:57.542329132 +0000 UTC m=+34.171983462 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-registry-tls") pod "image-registry-8c846c645-h9j62" (UID: "563c6d5b-c270-41de-bed4-b2aeb8fe8f42") : secret "image-registry-tls" not found
Sep 15 16:26:57.042630 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.042606    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-registry-certificates\") pod \"image-registry-8c846c645-h9j62\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") " pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:26:57.043071 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.043045    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8d780ae3-adfc-447b-9605-30793d9fc5c1-config-volume\") pod \"dns-default-45s5g\" (UID: \"8d780ae3-adfc-447b-9605-30793d9fc5c1\") " pod="openshift-dns/dns-default-45s5g"
Sep 15 16:26:57.043695 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.043317    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-data-volume\") pod \"insights-runtime-extractor-27hdl\" (UID: \"abc7e760-ef1c-48bd-8d65-c5ad19f3f422\") " pod="openshift-insights/insights-runtime-extractor-27hdl"
Sep 15 16:26:57.043695 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.043370    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d780ae3-adfc-447b-9605-30793d9fc5c1-tmp-dir\") pod \"dns-default-45s5g\" (UID: \"8d780ae3-adfc-447b-9605-30793d9fc5c1\") " pod="openshift-dns/dns-default-45s5g"
Sep 15 16:26:57.043695 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.043503    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-crio-socket\") pod \"insights-runtime-extractor-27hdl\" (UID: \"abc7e760-ef1c-48bd-8d65-c5ad19f3f422\") " pod="openshift-insights/insights-runtime-extractor-27hdl"
Sep 15 16:26:57.043695 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:57.043575    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 16:26:57.043695 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:57.043630    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-insights-runtime-extractor-tls podName:abc7e760-ef1c-48bd-8d65-c5ad19f3f422 nodeName:}" failed. No retries permitted until 2026-09-15 16:26:57.543614181 +0000 UTC m=+34.173268533 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-insights-runtime-extractor-tls") pod "insights-runtime-extractor-27hdl" (UID: "abc7e760-ef1c-48bd-8d65-c5ad19f3f422") : secret "insights-runtime-extractor-tls" not found
Sep 15 16:26:57.046820 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.046796    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-installation-pull-secrets\") pod \"image-registry-8c846c645-h9j62\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") " pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:26:57.046911 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.046889    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/84da0f72-32d0-4f59-9cad-f2c59cc71a0b-cert\") pod \"ingress-canary-2kxsk\" (UID: \"84da0f72-32d0-4f59-9cad-f2c59cc71a0b\") " pod="openshift-ingress-canary/ingress-canary-2kxsk"
Sep 15 16:26:57.051186 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.051167    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65szf\" (UniqueName: \"kubernetes.io/projected/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-kube-api-access-65szf\") pod \"insights-runtime-extractor-27hdl\" (UID: \"abc7e760-ef1c-48bd-8d65-c5ad19f3f422\") " pod="openshift-insights/insights-runtime-extractor-27hdl"
Sep 15 16:26:57.051290 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.051247    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4r6b\" (UniqueName: \"kubernetes.io/projected/8d780ae3-adfc-447b-9605-30793d9fc5c1-kube-api-access-x4r6b\") pod \"dns-default-45s5g\" (UID: \"8d780ae3-adfc-447b-9605-30793d9fc5c1\") " pod="openshift-dns/dns-default-45s5g"
Sep 15 16:26:57.051379 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.051358    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct8fd\" (UniqueName: \"kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-kube-api-access-ct8fd\") pod \"image-registry-8c846c645-h9j62\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") " pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:26:57.051379 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.051369    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgh7d\" (UniqueName: \"kubernetes.io/projected/84da0f72-32d0-4f59-9cad-f2c59cc71a0b-kube-api-access-rgh7d\") pod \"ingress-canary-2kxsk\" (UID: \"84da0f72-32d0-4f59-9cad-f2c59cc71a0b\") " pod="openshift-ingress-canary/ingress-canary-2kxsk"
Sep 15 16:26:57.051646 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.051627    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-image-registry-private-configuration\") pod \"image-registry-8c846c645-h9j62\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") " pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:26:57.053656 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.053635    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-bound-sa-token\") pod \"image-registry-8c846c645-h9j62\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") " pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:26:57.139106 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.139078    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2kxsk"
Sep 15 16:26:57.246537 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.246520    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8zk2f_f89021a6-bc8a-4f31-a76c-87c39f332c20/node-ca/0.log"
Sep 15 16:26:57.364447 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.364388    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-gjskw"]
Sep 15 16:26:57.381449 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.381424    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-gjskw"]
Sep 15 16:26:57.381582 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.381527    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-gjskw"
Sep 15 16:26:57.384233 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.384213    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 15 16:26:57.384944 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.384924    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 15 16:26:57.385078 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.385056    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-9ps7l\""
Sep 15 16:26:57.385288 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.385264    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 15 16:26:57.385334 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.384924    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 15 16:26:57.546535 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.546500    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-27hdl\" (UID: \"abc7e760-ef1c-48bd-8d65-c5ad19f3f422\") " pod="openshift-insights/insights-runtime-extractor-27hdl"
Sep 15 16:26:57.546535 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.546541    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-registry-tls\") pod \"image-registry-8c846c645-h9j62\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") " pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:26:57.546754 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.546571    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-27hdl\" (UID: \"abc7e760-ef1c-48bd-8d65-c5ad19f3f422\") " pod="openshift-insights/insights-runtime-extractor-27hdl"
Sep 15 16:26:57.546754 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.546611    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0426785e-0d95-4b9d-abda-8233fc3cfea6-signing-cabundle\") pod \"service-ca-568869fdc5-gjskw\" (UID: \"0426785e-0d95-4b9d-abda-8233fc3cfea6\") " pod="openshift-service-ca/service-ca-568869fdc5-gjskw"
Sep 15 16:26:57.546754 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.546634    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0426785e-0d95-4b9d-abda-8233fc3cfea6-signing-key\") pod \"service-ca-568869fdc5-gjskw\" (UID: \"0426785e-0d95-4b9d-abda-8233fc3cfea6\") " pod="openshift-service-ca/service-ca-568869fdc5-gjskw"
Sep 15 16:26:57.546754 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:57.546645    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 16:26:57.546754 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:57.546670    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 16:26:57.546754 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:57.546688    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8c846c645-h9j62: secret "image-registry-tls" not found
Sep 15 16:26:57.546754 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:57.546714    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-kube-rbac-proxy-cm podName:abc7e760-ef1c-48bd-8d65-c5ad19f3f422 nodeName:}" failed. No retries permitted until 2026-09-15 16:26:58.546695027 +0000 UTC m=+35.176349377 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-kube-rbac-proxy-cm") pod "insights-runtime-extractor-27hdl" (UID: "abc7e760-ef1c-48bd-8d65-c5ad19f3f422") : configmap "kube-rbac-proxy" not found
Sep 15 16:26:57.546754 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:57.546731    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 16:26:57.546754 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:57.546739    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-registry-tls podName:563c6d5b-c270-41de-bed4-b2aeb8fe8f42 nodeName:}" failed. No retries permitted until 2026-09-15 16:26:58.546722057 +0000 UTC m=+35.176376391 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-registry-tls") pod "image-registry-8c846c645-h9j62" (UID: "563c6d5b-c270-41de-bed4-b2aeb8fe8f42") : secret "image-registry-tls" not found
Sep 15 16:26:57.547222 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:57.546769    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-insights-runtime-extractor-tls podName:abc7e760-ef1c-48bd-8d65-c5ad19f3f422 nodeName:}" failed. No retries permitted until 2026-09-15 16:26:58.546757974 +0000 UTC m=+35.176412338 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-insights-runtime-extractor-tls") pod "insights-runtime-extractor-27hdl" (UID: "abc7e760-ef1c-48bd-8d65-c5ad19f3f422") : secret "insights-runtime-extractor-tls" not found
Sep 15 16:26:57.547222 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.546835    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8d780ae3-adfc-447b-9605-30793d9fc5c1-metrics-tls\") pod \"dns-default-45s5g\" (UID: \"8d780ae3-adfc-447b-9605-30793d9fc5c1\") " pod="openshift-dns/dns-default-45s5g"
Sep 15 16:26:57.547222 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.546865    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4xs7\" (UniqueName: \"kubernetes.io/projected/0426785e-0d95-4b9d-abda-8233fc3cfea6-kube-api-access-q4xs7\") pod \"service-ca-568869fdc5-gjskw\" (UID: \"0426785e-0d95-4b9d-abda-8233fc3cfea6\") " pod="openshift-service-ca/service-ca-568869fdc5-gjskw"
Sep 15 16:26:57.547222 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:57.546947    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 16:26:57.547222 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:57.547003    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d780ae3-adfc-447b-9605-30793d9fc5c1-metrics-tls podName:8d780ae3-adfc-447b-9605-30793d9fc5c1 nodeName:}" failed. No retries permitted until 2026-09-15 16:26:58.546990459 +0000 UTC m=+35.176644789 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8d780ae3-adfc-447b-9605-30793d9fc5c1-metrics-tls") pod "dns-default-45s5g" (UID: "8d780ae3-adfc-447b-9605-30793d9fc5c1") : secret "dns-default-metrics-tls" not found
Sep 15 16:26:57.647774 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.647696    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0426785e-0d95-4b9d-abda-8233fc3cfea6-signing-cabundle\") pod \"service-ca-568869fdc5-gjskw\" (UID: \"0426785e-0d95-4b9d-abda-8233fc3cfea6\") " pod="openshift-service-ca/service-ca-568869fdc5-gjskw"
Sep 15 16:26:57.647774 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.647745    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0426785e-0d95-4b9d-abda-8233fc3cfea6-signing-key\") pod \"service-ca-568869fdc5-gjskw\" (UID: \"0426785e-0d95-4b9d-abda-8233fc3cfea6\") " pod="openshift-service-ca/service-ca-568869fdc5-gjskw"
Sep 15 16:26:57.648001 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.647869    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4xs7\" (UniqueName: \"kubernetes.io/projected/0426785e-0d95-4b9d-abda-8233fc3cfea6-kube-api-access-q4xs7\") pod \"service-ca-568869fdc5-gjskw\" (UID: \"0426785e-0d95-4b9d-abda-8233fc3cfea6\") " pod="openshift-service-ca/service-ca-568869fdc5-gjskw"
Sep 15 16:26:57.648457 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.648426    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0426785e-0d95-4b9d-abda-8233fc3cfea6-signing-cabundle\") pod \"service-ca-568869fdc5-gjskw\" (UID: \"0426785e-0d95-4b9d-abda-8233fc3cfea6\") " pod="openshift-service-ca/service-ca-568869fdc5-gjskw"
Sep 15 16:26:57.650900 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.650882    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0426785e-0d95-4b9d-abda-8233fc3cfea6-signing-key\") pod \"service-ca-568869fdc5-gjskw\" (UID: \"0426785e-0d95-4b9d-abda-8233fc3cfea6\") " pod="openshift-service-ca/service-ca-568869fdc5-gjskw"
Sep 15 16:26:57.657185 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.657161    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4xs7\" (UniqueName: \"kubernetes.io/projected/0426785e-0d95-4b9d-abda-8233fc3cfea6-kube-api-access-q4xs7\") pod \"service-ca-568869fdc5-gjskw\" (UID: \"0426785e-0d95-4b9d-abda-8233fc3cfea6\") " pod="openshift-service-ca/service-ca-568869fdc5-gjskw"
Sep 15 16:26:57.693106 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:57.693079    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-gjskw"
Sep 15 16:26:58.274351 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:58.274164    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-gjskw"]
Sep 15 16:26:58.277583 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:58.277556    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2kxsk"]
Sep 15 16:26:58.277910 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:58.277889    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0426785e_0d95_4b9d_abda_8233fc3cfea6.slice/crio-671dbae0455a4577cf6f3c19063800bd934778a9400052a95f1c4833db9ea1b5 WatchSource:0}: Error finding container 671dbae0455a4577cf6f3c19063800bd934778a9400052a95f1c4833db9ea1b5: Status 404 returned error can't find the container with id 671dbae0455a4577cf6f3c19063800bd934778a9400052a95f1c4833db9ea1b5
Sep 15 16:26:58.280179 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:26:58.280151    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84da0f72_32d0_4f59_9cad_f2c59cc71a0b.slice/crio-ded757100340b9c582f3ee0cdd1f928e25575ba58d4409e23c8a8fa86b7cc10a WatchSource:0}: Error finding container ded757100340b9c582f3ee0cdd1f928e25575ba58d4409e23c8a8fa86b7cc10a: Status 404 returned error can't find the container with id ded757100340b9c582f3ee0cdd1f928e25575ba58d4409e23c8a8fa86b7cc10a
Sep 15 16:26:58.553913 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:58.553887    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8d780ae3-adfc-447b-9605-30793d9fc5c1-metrics-tls\") pod \"dns-default-45s5g\" (UID: \"8d780ae3-adfc-447b-9605-30793d9fc5c1\") " pod="openshift-dns/dns-default-45s5g"
Sep 15 16:26:58.554419 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:58.553925    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-27hdl\" (UID: \"abc7e760-ef1c-48bd-8d65-c5ad19f3f422\") " pod="openshift-insights/insights-runtime-extractor-27hdl"
Sep 15 16:26:58.554419 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:58.553944    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-registry-tls\") pod \"image-registry-8c846c645-h9j62\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") " pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:26:58.554419 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:58.553973    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-27hdl\" (UID: \"abc7e760-ef1c-48bd-8d65-c5ad19f3f422\") " pod="openshift-insights/insights-runtime-extractor-27hdl"
Sep 15 16:26:58.554419 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:58.554026    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 16:26:58.554419 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:58.554037    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 16:26:58.554419 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:58.554047    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 16:26:58.554419 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:58.554066    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8c846c645-h9j62: secret "image-registry-tls" not found
Sep 15 16:26:58.554419 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:58.554075    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 16:26:58.554419 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:58.554094    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-kube-rbac-proxy-cm podName:abc7e760-ef1c-48bd-8d65-c5ad19f3f422 nodeName:}" failed. No retries permitted until 2026-09-15 16:27:00.554076109 +0000 UTC m=+37.183730452 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-kube-rbac-proxy-cm") pod "insights-runtime-extractor-27hdl" (UID: "abc7e760-ef1c-48bd-8d65-c5ad19f3f422") : configmap "kube-rbac-proxy" not found
Sep 15 16:26:58.554419 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:58.554113    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-insights-runtime-extractor-tls podName:abc7e760-ef1c-48bd-8d65-c5ad19f3f422 nodeName:}" failed. No retries permitted until 2026-09-15 16:27:00.554101208 +0000 UTC m=+37.183755539 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-insights-runtime-extractor-tls") pod "insights-runtime-extractor-27hdl" (UID: "abc7e760-ef1c-48bd-8d65-c5ad19f3f422") : secret "insights-runtime-extractor-tls" not found
Sep 15 16:26:58.554419 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:58.554128    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d780ae3-adfc-447b-9605-30793d9fc5c1-metrics-tls podName:8d780ae3-adfc-447b-9605-30793d9fc5c1 nodeName:}" failed. No retries permitted until 2026-09-15 16:27:00.554119789 +0000 UTC m=+37.183774121 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8d780ae3-adfc-447b-9605-30793d9fc5c1-metrics-tls") pod "dns-default-45s5g" (UID: "8d780ae3-adfc-447b-9605-30793d9fc5c1") : secret "dns-default-metrics-tls" not found
Sep 15 16:26:58.554419 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:26:58.554141    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-registry-tls podName:563c6d5b-c270-41de-bed4-b2aeb8fe8f42 nodeName:}" failed. No retries permitted until 2026-09-15 16:27:00.554133107 +0000 UTC m=+37.183787437 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-registry-tls") pod "image-registry-8c846c645-h9j62" (UID: "563c6d5b-c270-41de-bed4-b2aeb8fe8f42") : secret "image-registry-tls" not found
Sep 15 16:26:59.068072 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:59.068042    2579 generic.go:356] "Generic (PLEG): container finished" podID="b17d9664-4ce9-449a-8826-af717fd47f97" containerID="1ce71fe8ac9c050cdb80403deef086fdf51825d58d72f1aebe456d96561e2ed6" exitCode=0
Sep 15 16:26:59.068277 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:59.068124    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cvgk4" event={"ID":"b17d9664-4ce9-449a-8826-af717fd47f97","Type":"ContainerDied","Data":"1ce71fe8ac9c050cdb80403deef086fdf51825d58d72f1aebe456d96561e2ed6"}
Sep 15 16:26:59.069643 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:59.069608    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2kxsk" event={"ID":"84da0f72-32d0-4f59-9cad-f2c59cc71a0b","Type":"ContainerStarted","Data":"ded757100340b9c582f3ee0cdd1f928e25575ba58d4409e23c8a8fa86b7cc10a"}
Sep 15 16:26:59.070874 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:26:59.070848    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-gjskw" event={"ID":"0426785e-0d95-4b9d-abda-8233fc3cfea6","Type":"ContainerStarted","Data":"671dbae0455a4577cf6f3c19063800bd934778a9400052a95f1c4833db9ea1b5"}
Sep 15 16:27:00.078558 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:00.078511    2579 generic.go:356] "Generic (PLEG): container finished" podID="b17d9664-4ce9-449a-8826-af717fd47f97" containerID="91d6faca71aa3d733ea0a6502560d6ea70539cdbd52ac80a1c0aad5c08ef209b" exitCode=0
Sep 15 16:27:00.079342 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:00.078550    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cvgk4" event={"ID":"b17d9664-4ce9-449a-8826-af717fd47f97","Type":"ContainerDied","Data":"91d6faca71aa3d733ea0a6502560d6ea70539cdbd52ac80a1c0aad5c08ef209b"}
Sep 15 16:27:00.570944 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:00.570916    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8d780ae3-adfc-447b-9605-30793d9fc5c1-metrics-tls\") pod \"dns-default-45s5g\" (UID: \"8d780ae3-adfc-447b-9605-30793d9fc5c1\") " pod="openshift-dns/dns-default-45s5g"
Sep 15 16:27:00.571092 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:00.570951    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-27hdl\" (UID: \"abc7e760-ef1c-48bd-8d65-c5ad19f3f422\") " pod="openshift-insights/insights-runtime-extractor-27hdl"
Sep 15 16:27:00.571092 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:00.570974    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-registry-tls\") pod \"image-registry-8c846c645-h9j62\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") " pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:27:00.571092 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:00.570992    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-27hdl\" (UID: \"abc7e760-ef1c-48bd-8d65-c5ad19f3f422\") " pod="openshift-insights/insights-runtime-extractor-27hdl"
Sep 15 16:27:00.571092 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:27:00.571047    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 16:27:00.571092 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:27:00.571074    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 16:27:00.571092 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:27:00.571086    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 16:27:00.571092 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:27:00.571090    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 16:27:00.571298 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:27:00.571105    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8c846c645-h9j62: secret "image-registry-tls" not found
Sep 15 16:27:00.571298 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:27:00.571112    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-kube-rbac-proxy-cm podName:abc7e760-ef1c-48bd-8d65-c5ad19f3f422 nodeName:}" failed. No retries permitted until 2026-09-15 16:27:04.57109624 +0000 UTC m=+41.200750573 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-kube-rbac-proxy-cm") pod "insights-runtime-extractor-27hdl" (UID: "abc7e760-ef1c-48bd-8d65-c5ad19f3f422") : configmap "kube-rbac-proxy" not found
Sep 15 16:27:00.571298 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:27:00.571131    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d780ae3-adfc-447b-9605-30793d9fc5c1-metrics-tls podName:8d780ae3-adfc-447b-9605-30793d9fc5c1 nodeName:}" failed. No retries permitted until 2026-09-15 16:27:04.571118181 +0000 UTC m=+41.200772514 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8d780ae3-adfc-447b-9605-30793d9fc5c1-metrics-tls") pod "dns-default-45s5g" (UID: "8d780ae3-adfc-447b-9605-30793d9fc5c1") : secret "dns-default-metrics-tls" not found
Sep 15 16:27:00.571298 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:27:00.571146    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-insights-runtime-extractor-tls podName:abc7e760-ef1c-48bd-8d65-c5ad19f3f422 nodeName:}" failed. No retries permitted until 2026-09-15 16:27:04.571139905 +0000 UTC m=+41.200794235 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-insights-runtime-extractor-tls") pod "insights-runtime-extractor-27hdl" (UID: "abc7e760-ef1c-48bd-8d65-c5ad19f3f422") : secret "insights-runtime-extractor-tls" not found
Sep 15 16:27:00.571298 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:27:00.571162    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-registry-tls podName:563c6d5b-c270-41de-bed4-b2aeb8fe8f42 nodeName:}" failed. No retries permitted until 2026-09-15 16:27:04.571153287 +0000 UTC m=+41.200807617 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-registry-tls") pod "image-registry-8c846c645-h9j62" (UID: "563c6d5b-c270-41de-bed4-b2aeb8fe8f42") : secret "image-registry-tls" not found
Sep 15 16:27:02.086625 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:02.086324    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cvgk4" event={"ID":"b17d9664-4ce9-449a-8826-af717fd47f97","Type":"ContainerStarted","Data":"81bf4ec7cfde17db728cfddffc5cb5cd554d4fe1ee95dedb4aa20e979d8e123a"}
Sep 15 16:27:02.088073 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:02.088044    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2kxsk" event={"ID":"84da0f72-32d0-4f59-9cad-f2c59cc71a0b","Type":"ContainerStarted","Data":"61a63c36c67f5eb10f3f0f43a101915ff34254b7f3bbe2042a397730ddc4c5ef"}
Sep 15 16:27:02.089485 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:02.089459    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-gjskw" event={"ID":"0426785e-0d95-4b9d-abda-8233fc3cfea6","Type":"ContainerStarted","Data":"117b10c60bf84ddba8ea3c83d134ab75ec7e962510a790954da699bc3aef97bd"}
Sep 15 16:27:02.127895 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:02.127856    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-cvgk4" podStartSLOduration=8.562597382 podStartE2EDuration="39.127841609s" totalImagesPullingTime="30.565244227s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 16:26:23 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:27:02.109995636 +0000 UTC m=+38.739649988" watchObservedRunningTime="2026-09-15 16:27:02.127841609 +0000 UTC m=+38.757495957"
Sep 15 16:27:02.127982 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:02.127960    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-gjskw" podStartSLOduration=1.6482912920000001 podStartE2EDuration="5.127954919s" totalImagesPullingTime="3.479663627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 16:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:27:02.127708654 +0000 UTC m=+38.757363008" watchObservedRunningTime="2026-09-15 16:27:02.127954919 +0000 UTC m=+38.757609261"
Sep 15 16:27:02.150364 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:02.150332    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2kxsk" podStartSLOduration=2.672488876 podStartE2EDuration="6.150320586s" totalImagesPullingTime="3.47783171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 16:26:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:27:02.149874602 +0000 UTC m=+38.779528953" watchObservedRunningTime="2026-09-15 16:27:02.150320586 +0000 UTC m=+38.779974931"
Sep 15 16:27:04.599172 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:04.599133    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8d780ae3-adfc-447b-9605-30793d9fc5c1-metrics-tls\") pod \"dns-default-45s5g\" (UID: \"8d780ae3-adfc-447b-9605-30793d9fc5c1\") " pod="openshift-dns/dns-default-45s5g"
Sep 15 16:27:04.599556 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:04.599188    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-27hdl\" (UID: \"abc7e760-ef1c-48bd-8d65-c5ad19f3f422\") " pod="openshift-insights/insights-runtime-extractor-27hdl"
Sep 15 16:27:04.599556 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:04.599218    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-registry-tls\") pod \"image-registry-8c846c645-h9j62\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") " pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:27:04.599556 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:04.599246    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-27hdl\" (UID: \"abc7e760-ef1c-48bd-8d65-c5ad19f3f422\") " pod="openshift-insights/insights-runtime-extractor-27hdl"
Sep 15 16:27:04.599556 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:27:04.599290    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 16:27:04.599556 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:27:04.599328    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 16:27:04.599556 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:27:04.599354    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d780ae3-adfc-447b-9605-30793d9fc5c1-metrics-tls podName:8d780ae3-adfc-447b-9605-30793d9fc5c1 nodeName:}" failed. No retries permitted until 2026-09-15 16:27:12.599334297 +0000 UTC m=+49.228988641 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8d780ae3-adfc-447b-9605-30793d9fc5c1-metrics-tls") pod "dns-default-45s5g" (UID: "8d780ae3-adfc-447b-9605-30793d9fc5c1") : secret "dns-default-metrics-tls" not found
Sep 15 16:27:04.599556 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:27:04.599362    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 16:27:04.599556 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:27:04.599371    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-kube-rbac-proxy-cm podName:abc7e760-ef1c-48bd-8d65-c5ad19f3f422 nodeName:}" failed. No retries permitted until 2026-09-15 16:27:12.599362169 +0000 UTC m=+49.229016502 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-kube-rbac-proxy-cm") pod "insights-runtime-extractor-27hdl" (UID: "abc7e760-ef1c-48bd-8d65-c5ad19f3f422") : configmap "kube-rbac-proxy" not found
Sep 15 16:27:04.599556 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:27:04.599374    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8c846c645-h9j62: secret "image-registry-tls" not found
Sep 15 16:27:04.599556 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:27:04.599421    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-registry-tls podName:563c6d5b-c270-41de-bed4-b2aeb8fe8f42 nodeName:}" failed. No retries permitted until 2026-09-15 16:27:12.599406943 +0000 UTC m=+49.229061292 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-registry-tls") pod "image-registry-8c846c645-h9j62" (UID: "563c6d5b-c270-41de-bed4-b2aeb8fe8f42") : secret "image-registry-tls" not found
Sep 15 16:27:04.599556 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:27:04.599436    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 16:27:04.599556 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:27:04.599484    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-insights-runtime-extractor-tls podName:abc7e760-ef1c-48bd-8d65-c5ad19f3f422 nodeName:}" failed. No retries permitted until 2026-09-15 16:27:12.599470227 +0000 UTC m=+49.229124557 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-insights-runtime-extractor-tls") pod "insights-runtime-extractor-27hdl" (UID: "abc7e760-ef1c-48bd-8d65-c5ad19f3f422") : secret "insights-runtime-extractor-tls" not found
Sep 15 16:27:12.658627 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:12.658595    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-27hdl\" (UID: \"abc7e760-ef1c-48bd-8d65-c5ad19f3f422\") " pod="openshift-insights/insights-runtime-extractor-27hdl"
Sep 15 16:27:12.658627 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:12.658630    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-registry-tls\") pod \"image-registry-8c846c645-h9j62\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") " pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:27:12.659137 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:12.658649    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-27hdl\" (UID: \"abc7e760-ef1c-48bd-8d65-c5ad19f3f422\") " pod="openshift-insights/insights-runtime-extractor-27hdl"
Sep 15 16:27:12.659137 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:27:12.658743    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 16:27:12.659137 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:12.658766    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8d780ae3-adfc-447b-9605-30793d9fc5c1-metrics-tls\") pod \"dns-default-45s5g\" (UID: \"8d780ae3-adfc-447b-9605-30793d9fc5c1\") " pod="openshift-dns/dns-default-45s5g"
Sep 15 16:27:12.659137 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:27:12.658814    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-insights-runtime-extractor-tls podName:abc7e760-ef1c-48bd-8d65-c5ad19f3f422 nodeName:}" failed. No retries permitted until 2026-09-15 16:27:28.658798746 +0000 UTC m=+65.288453092 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-insights-runtime-extractor-tls") pod "insights-runtime-extractor-27hdl" (UID: "abc7e760-ef1c-48bd-8d65-c5ad19f3f422") : secret "insights-runtime-extractor-tls" not found
Sep 15 16:27:12.659296 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:12.659202    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-27hdl\" (UID: \"abc7e760-ef1c-48bd-8d65-c5ad19f3f422\") " pod="openshift-insights/insights-runtime-extractor-27hdl"
Sep 15 16:27:12.661845 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:12.661824    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8d780ae3-adfc-447b-9605-30793d9fc5c1-metrics-tls\") pod \"dns-default-45s5g\" (UID: \"8d780ae3-adfc-447b-9605-30793d9fc5c1\") " pod="openshift-dns/dns-default-45s5g"
Sep 15 16:27:12.661942 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:12.661852    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-registry-tls\") pod \"image-registry-8c846c645-h9j62\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") " pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:27:12.697373 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:12.697353    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:27:12.727761 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:12.727741    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-45s5g"
Sep 15 16:27:13.019140 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:13.019117    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8c846c645-h9j62"]
Sep 15 16:27:13.019926 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:27:13.019905    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod563c6d5b_c270_41de_bed4_b2aeb8fe8f42.slice/crio-21819786c57d326bbf1d876996597f9dfa3058b3ca4a0fa82f02eb439f11dd37 WatchSource:0}: Error finding container 21819786c57d326bbf1d876996597f9dfa3058b3ca4a0fa82f02eb439f11dd37: Status 404 returned error can't find the container with id 21819786c57d326bbf1d876996597f9dfa3058b3ca4a0fa82f02eb439f11dd37
Sep 15 16:27:13.095035 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:13.095011    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-45s5g"]
Sep 15 16:27:13.097645 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:27:13.097622    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d780ae3_adfc_447b_9605_30793d9fc5c1.slice/crio-749a3398e7a46a5aae459c51dde49842af4777163fcec9ce701de6d2bca0f647 WatchSource:0}: Error finding container 749a3398e7a46a5aae459c51dde49842af4777163fcec9ce701de6d2bca0f647: Status 404 returned error can't find the container with id 749a3398e7a46a5aae459c51dde49842af4777163fcec9ce701de6d2bca0f647
Sep 15 16:27:13.115295 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:13.115268    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8c846c645-h9j62" event={"ID":"563c6d5b-c270-41de-bed4-b2aeb8fe8f42","Type":"ContainerStarted","Data":"21819786c57d326bbf1d876996597f9dfa3058b3ca4a0fa82f02eb439f11dd37"}
Sep 15 16:27:13.116345 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:13.116315    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-45s5g" event={"ID":"8d780ae3-adfc-447b-9605-30793d9fc5c1","Type":"ContainerStarted","Data":"749a3398e7a46a5aae459c51dde49842af4777163fcec9ce701de6d2bca0f647"}
Sep 15 16:27:14.120292 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:14.120259    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8c846c645-h9j62" event={"ID":"563c6d5b-c270-41de-bed4-b2aeb8fe8f42","Type":"ContainerStarted","Data":"9e81120a09b5c95595aef12003136ba87f07d224f361b6a246b96333a251ff59"}
Sep 15 16:27:14.120677 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:14.120431    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:27:15.125894 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:15.125869    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-45s5g" event={"ID":"8d780ae3-adfc-447b-9605-30793d9fc5c1","Type":"ContainerStarted","Data":"2834dbbf11a9ce68dcc3fe57fa48a9e8107c7f088a3efeb4a708b67039e02e3f"}
Sep 15 16:27:16.130009 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:16.129978    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-45s5g" event={"ID":"8d780ae3-adfc-447b-9605-30793d9fc5c1","Type":"ContainerStarted","Data":"83dfba84516bd2b47a5d36e385b6d7dfa815c4ebb3a917c62de3681151342d7e"}
Sep 15 16:27:16.130400 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:16.130117    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-45s5g"
Sep 15 16:27:16.148327 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:16.148283    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-8c846c645-h9j62" podStartSLOduration=55.148270152 podStartE2EDuration="55.148270152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 16:26:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:27:14.141412107 +0000 UTC m=+50.771066458" watchObservedRunningTime="2026-09-15 16:27:16.148270152 +0000 UTC m=+52.777924503"
Sep 15 16:27:16.148645 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:16.148618    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-45s5g" podStartSLOduration=18.35451761 podStartE2EDuration="20.14860865s" totalImagesPullingTime="1.79409104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 16:26:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:27:16.147828176 +0000 UTC m=+52.777482528" watchObservedRunningTime="2026-09-15 16:27:16.14860865 +0000 UTC m=+52.778263001"
Sep 15 16:27:20.136162 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:20.136133    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8c846c645-h9j62"]
Sep 15 16:27:20.183843 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:20.183817    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8565fb9966-gk2rg"]
Sep 15 16:27:20.188915 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:20.188897    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8565fb9966-gk2rg"
Sep 15 16:27:20.206956 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:20.206934    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0aedf5a4-039b-42c4-8659-bc6e047eebe0-registry-tls\") pod \"image-registry-8565fb9966-gk2rg\" (UID: \"0aedf5a4-039b-42c4-8659-bc6e047eebe0\") " pod="openshift-image-registry/image-registry-8565fb9966-gk2rg"
Sep 15 16:27:20.207059 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:20.206971    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0aedf5a4-039b-42c4-8659-bc6e047eebe0-trusted-ca\") pod \"image-registry-8565fb9966-gk2rg\" (UID: \"0aedf5a4-039b-42c4-8659-bc6e047eebe0\") " pod="openshift-image-registry/image-registry-8565fb9966-gk2rg"
Sep 15 16:27:20.207059 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:20.206987    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0aedf5a4-039b-42c4-8659-bc6e047eebe0-bound-sa-token\") pod \"image-registry-8565fb9966-gk2rg\" (UID: \"0aedf5a4-039b-42c4-8659-bc6e047eebe0\") " pod="openshift-image-registry/image-registry-8565fb9966-gk2rg"
Sep 15 16:27:20.207059 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:20.207008    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0aedf5a4-039b-42c4-8659-bc6e047eebe0-registry-certificates\") pod \"image-registry-8565fb9966-gk2rg\" (UID: \"0aedf5a4-039b-42c4-8659-bc6e047eebe0\") " pod="openshift-image-registry/image-registry-8565fb9966-gk2rg"
Sep 15 16:27:20.207214 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:20.207111    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf9rt\" (UniqueName: \"kubernetes.io/projected/0aedf5a4-039b-42c4-8659-bc6e047eebe0-kube-api-access-gf9rt\") pod \"image-registry-8565fb9966-gk2rg\" (UID: \"0aedf5a4-039b-42c4-8659-bc6e047eebe0\") " pod="openshift-image-registry/image-registry-8565fb9966-gk2rg"
Sep 15 16:27:20.207214 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:20.207145    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0aedf5a4-039b-42c4-8659-bc6e047eebe0-installation-pull-secrets\") pod \"image-registry-8565fb9966-gk2rg\" (UID: \"0aedf5a4-039b-42c4-8659-bc6e047eebe0\") " pod="openshift-image-registry/image-registry-8565fb9966-gk2rg"
Sep 15 16:27:20.207214 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:20.207170    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0aedf5a4-039b-42c4-8659-bc6e047eebe0-ca-trust-extracted\") pod \"image-registry-8565fb9966-gk2rg\" (UID: \"0aedf5a4-039b-42c4-8659-bc6e047eebe0\") " pod="openshift-image-registry/image-registry-8565fb9966-gk2rg"
Sep 15 16:27:20.207332 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:20.207214    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0aedf5a4-039b-42c4-8659-bc6e047eebe0-image-registry-private-configuration\") pod \"image-registry-8565fb9966-gk2rg\" (UID: \"0aedf5a4-039b-42c4-8659-bc6e047eebe0\") " pod="openshift-image-registry/image-registry-8565fb9966-gk2rg"
Sep 15 16:27:20.217562 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:20.217542    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8565fb9966-gk2rg"]
Sep 15 16:27:20.307528 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:20.307500    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0aedf5a4-039b-42c4-8659-bc6e047eebe0-registry-certificates\") pod \"image-registry-8565fb9966-gk2rg\" (UID: \"0aedf5a4-039b-42c4-8659-bc6e047eebe0\") " pod="openshift-image-registry/image-registry-8565fb9966-gk2rg"
Sep 15 16:27:20.307609 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:20.307556    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf9rt\" (UniqueName: \"kubernetes.io/projected/0aedf5a4-039b-42c4-8659-bc6e047eebe0-kube-api-access-gf9rt\") pod \"image-registry-8565fb9966-gk2rg\" (UID: \"0aedf5a4-039b-42c4-8659-bc6e047eebe0\") " pod="openshift-image-registry/image-registry-8565fb9966-gk2rg"
Sep 15 16:27:20.307609 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:20.307573    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0aedf5a4-039b-42c4-8659-bc6e047eebe0-installation-pull-secrets\") pod \"image-registry-8565fb9966-gk2rg\" (UID: \"0aedf5a4-039b-42c4-8659-bc6e047eebe0\") " pod="openshift-image-registry/image-registry-8565fb9966-gk2rg"
Sep 15 16:27:20.307609 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:20.307591    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0aedf5a4-039b-42c4-8659-bc6e047eebe0-ca-trust-extracted\") pod \"image-registry-8565fb9966-gk2rg\" (UID: \"0aedf5a4-039b-42c4-8659-bc6e047eebe0\") " pod="openshift-image-registry/image-registry-8565fb9966-gk2rg"
Sep 15 16:27:20.307750 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:20.307621    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0aedf5a4-039b-42c4-8659-bc6e047eebe0-image-registry-private-configuration\") pod \"image-registry-8565fb9966-gk2rg\" (UID: \"0aedf5a4-039b-42c4-8659-bc6e047eebe0\") " pod="openshift-image-registry/image-registry-8565fb9966-gk2rg"
Sep 15 16:27:20.307750 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:20.307642    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0aedf5a4-039b-42c4-8659-bc6e047eebe0-registry-tls\") pod \"image-registry-8565fb9966-gk2rg\" (UID: \"0aedf5a4-039b-42c4-8659-bc6e047eebe0\") " pod="openshift-image-registry/image-registry-8565fb9966-gk2rg"
Sep 15 16:27:20.307750 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:20.307671    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0aedf5a4-039b-42c4-8659-bc6e047eebe0-trusted-ca\") pod \"image-registry-8565fb9966-gk2rg\" (UID: \"0aedf5a4-039b-42c4-8659-bc6e047eebe0\") " pod="openshift-image-registry/image-registry-8565fb9966-gk2rg"
Sep 15 16:27:20.307941 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:20.307920    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0aedf5a4-039b-42c4-8659-bc6e047eebe0-bound-sa-token\") pod \"image-registry-8565fb9966-gk2rg\" (UID: \"0aedf5a4-039b-42c4-8659-bc6e047eebe0\") " pod="openshift-image-registry/image-registry-8565fb9966-gk2rg"
Sep 15 16:27:20.308048 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:20.308033    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0aedf5a4-039b-42c4-8659-bc6e047eebe0-ca-trust-extracted\") pod \"image-registry-8565fb9966-gk2rg\" (UID: \"0aedf5a4-039b-42c4-8659-bc6e047eebe0\") " pod="openshift-image-registry/image-registry-8565fb9966-gk2rg"
Sep 15 16:27:20.308377 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:20.308354    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0aedf5a4-039b-42c4-8659-bc6e047eebe0-registry-certificates\") pod \"image-registry-8565fb9966-gk2rg\" (UID: \"0aedf5a4-039b-42c4-8659-bc6e047eebe0\") " pod="openshift-image-registry/image-registry-8565fb9966-gk2rg"
Sep 15 16:27:20.308633 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:20.308612    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0aedf5a4-039b-42c4-8659-bc6e047eebe0-trusted-ca\") pod \"image-registry-8565fb9966-gk2rg\" (UID: \"0aedf5a4-039b-42c4-8659-bc6e047eebe0\") " pod="openshift-image-registry/image-registry-8565fb9966-gk2rg"
Sep 15 16:27:20.310249 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:20.310226    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0aedf5a4-039b-42c4-8659-bc6e047eebe0-image-registry-private-configuration\") pod \"image-registry-8565fb9966-gk2rg\" (UID: \"0aedf5a4-039b-42c4-8659-bc6e047eebe0\") " pod="openshift-image-registry/image-registry-8565fb9966-gk2rg"
Sep 15 16:27:20.310711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:20.310691    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0aedf5a4-039b-42c4-8659-bc6e047eebe0-installation-pull-secrets\") pod \"image-registry-8565fb9966-gk2rg\" (UID: \"0aedf5a4-039b-42c4-8659-bc6e047eebe0\") " pod="openshift-image-registry/image-registry-8565fb9966-gk2rg"
Sep 15 16:27:20.310754 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:20.310691    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0aedf5a4-039b-42c4-8659-bc6e047eebe0-registry-tls\") pod \"image-registry-8565fb9966-gk2rg\" (UID: \"0aedf5a4-039b-42c4-8659-bc6e047eebe0\") " pod="openshift-image-registry/image-registry-8565fb9966-gk2rg"
Sep 15 16:27:20.316844 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:20.316823    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf9rt\" (UniqueName: \"kubernetes.io/projected/0aedf5a4-039b-42c4-8659-bc6e047eebe0-kube-api-access-gf9rt\") pod \"image-registry-8565fb9966-gk2rg\" (UID: \"0aedf5a4-039b-42c4-8659-bc6e047eebe0\") " pod="openshift-image-registry/image-registry-8565fb9966-gk2rg"
Sep 15 16:27:20.317326 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:20.317305    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0aedf5a4-039b-42c4-8659-bc6e047eebe0-bound-sa-token\") pod \"image-registry-8565fb9966-gk2rg\" (UID: \"0aedf5a4-039b-42c4-8659-bc6e047eebe0\") " pod="openshift-image-registry/image-registry-8565fb9966-gk2rg"
Sep 15 16:27:20.497458 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:20.497437    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8565fb9966-gk2rg"
Sep 15 16:27:20.798952 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:20.798893    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8565fb9966-gk2rg"]
Sep 15 16:27:20.802249 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:27:20.802225    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aedf5a4_039b_42c4_8659_bc6e047eebe0.slice/crio-8b356de7761e6797faaa30f001652ac9182be7a60dc54449ba0b4316e7e3f56f WatchSource:0}: Error finding container 8b356de7761e6797faaa30f001652ac9182be7a60dc54449ba0b4316e7e3f56f: Status 404 returned error can't find the container with id 8b356de7761e6797faaa30f001652ac9182be7a60dc54449ba0b4316e7e3f56f
Sep 15 16:27:21.149454 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:21.149378    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8565fb9966-gk2rg" event={"ID":"0aedf5a4-039b-42c4-8659-bc6e047eebe0","Type":"ContainerStarted","Data":"a2ddf3b88eac0d07391f7bff53b7fadb6b619422d94ae240f9f2bac19dc3dacb"}
Sep 15 16:27:21.149454 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:21.149428    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8565fb9966-gk2rg" event={"ID":"0aedf5a4-039b-42c4-8659-bc6e047eebe0","Type":"ContainerStarted","Data":"8b356de7761e6797faaa30f001652ac9182be7a60dc54449ba0b4316e7e3f56f"}
Sep 15 16:27:21.149951 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:21.149510    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8565fb9966-gk2rg"
Sep 15 16:27:22.065714 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:22.065688    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b4rdn"
Sep 15 16:27:22.092130 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:22.092083    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-8565fb9966-gk2rg" podStartSLOduration=2.092066896 podStartE2EDuration="2.092066896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 16:27:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:27:21.169987197 +0000 UTC m=+57.799641549" watchObservedRunningTime="2026-09-15 16:27:22.092066896 +0000 UTC m=+58.721721247"
Sep 15 16:27:26.136251 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:26.136220    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-45s5g"
Sep 15 16:27:28.554550 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:28.554516    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d768ac0-1487-4850-bd9b-c66f73533e97-metrics-certs\") pod \"network-metrics-daemon-txzqk\" (UID: \"1d768ac0-1487-4850-bd9b-c66f73533e97\") " pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:27:28.557179 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:28.557161    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 15 16:27:28.567765 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:28.567741    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d768ac0-1487-4850-bd9b-c66f73533e97-metrics-certs\") pod \"network-metrics-daemon-txzqk\" (UID: \"1d768ac0-1487-4850-bd9b-c66f73533e97\") " pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:27:28.654957 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:28.654932    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d1e544d2-21e5-4abd-82d9-33a82470f13c-original-pull-secret\") pod \"global-pull-secret-syncer-9hbcw\" (UID: \"d1e544d2-21e5-4abd-82d9-33a82470f13c\") " pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:27:28.657996 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:28.657981    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 15 16:27:28.667184 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:28.667165    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d1e544d2-21e5-4abd-82d9-33a82470f13c-original-pull-secret\") pod \"global-pull-secret-syncer-9hbcw\" (UID: \"d1e544d2-21e5-4abd-82d9-33a82470f13c\") " pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:27:28.722928 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:28.722909    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zqb5b\""
Sep 15 16:27:28.725956 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:28.725944    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hbcw"
Sep 15 16:27:28.731086 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:28.731073    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzqk"
Sep 15 16:27:28.755512 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:28.755493    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5h7q\" (UniqueName: \"kubernetes.io/projected/ad430be2-7c5f-4402-a5a7-c8b4359b8c02-kube-api-access-r5h7q\") pod \"network-check-target-sqdn2\" (UID: \"ad430be2-7c5f-4402-a5a7-c8b4359b8c02\") " pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:27:28.755597 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:28.755531    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-27hdl\" (UID: \"abc7e760-ef1c-48bd-8d65-c5ad19f3f422\") " pod="openshift-insights/insights-runtime-extractor-27hdl"
Sep 15 16:27:28.757583 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:28.757566    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/abc7e760-ef1c-48bd-8d65-c5ad19f3f422-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-27hdl\" (UID: \"abc7e760-ef1c-48bd-8d65-c5ad19f3f422\") " pod="openshift-insights/insights-runtime-extractor-27hdl"
Sep 15 16:27:28.758187 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:28.758172    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 15 16:27:28.786773 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:28.786745    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 15 16:27:28.794249 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:28.794231    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5h7q\" (UniqueName: \"kubernetes.io/projected/ad430be2-7c5f-4402-a5a7-c8b4359b8c02-kube-api-access-r5h7q\") pod \"network-check-target-sqdn2\" (UID: \"ad430be2-7c5f-4402-a5a7-c8b4359b8c02\") " pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:27:28.910006 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:28.909980    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zvqwt\""
Sep 15 16:27:28.917347 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:28.917330    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-27hdl"
Sep 15 16:27:29.016623 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:29.016605    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-v6gf7\""
Sep 15 16:27:29.025105 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:29.025086    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:27:29.041990 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:29.041968    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9hbcw"]
Sep 15 16:27:29.045166 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:27:29.045139    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1e544d2_21e5_4abd_82d9_33a82470f13c.slice/crio-cb24cd0c3ff20b0933e70b4bdc2b7cf05de0c680ceb9c0f8d401359e6d2c0627 WatchSource:0}: Error finding container cb24cd0c3ff20b0933e70b4bdc2b7cf05de0c680ceb9c0f8d401359e6d2c0627: Status 404 returned error can't find the container with id cb24cd0c3ff20b0933e70b4bdc2b7cf05de0c680ceb9c0f8d401359e6d2c0627
Sep 15 16:27:29.066638 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:29.066588    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-txzqk"]
Sep 15 16:27:29.069378 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:27:29.069353    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d768ac0_1487_4850_bd9b_c66f73533e97.slice/crio-d83131060228b7b8a5b81d53425aa879c0ee65fff81da52ea61fdec06b244a14 WatchSource:0}: Error finding container d83131060228b7b8a5b81d53425aa879c0ee65fff81da52ea61fdec06b244a14: Status 404 returned error can't find the container with id d83131060228b7b8a5b81d53425aa879c0ee65fff81da52ea61fdec06b244a14
Sep 15 16:27:29.173302 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:29.173271    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-txzqk" event={"ID":"1d768ac0-1487-4850-bd9b-c66f73533e97","Type":"ContainerStarted","Data":"d83131060228b7b8a5b81d53425aa879c0ee65fff81da52ea61fdec06b244a14"}
Sep 15 16:27:29.174402 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:29.174381    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9hbcw" event={"ID":"d1e544d2-21e5-4abd-82d9-33a82470f13c","Type":"ContainerStarted","Data":"cb24cd0c3ff20b0933e70b4bdc2b7cf05de0c680ceb9c0f8d401359e6d2c0627"}
Sep 15 16:27:29.204575 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:29.204550    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-27hdl"]
Sep 15 16:27:29.207844 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:27:29.207823    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabc7e760_ef1c_48bd_8d65_c5ad19f3f422.slice/crio-37eb7365c18ff1de6440f80969dd9c81ab13b2388980a6a4a60cfc63cd7cc07f WatchSource:0}: Error finding container 37eb7365c18ff1de6440f80969dd9c81ab13b2388980a6a4a60cfc63cd7cc07f: Status 404 returned error can't find the container with id 37eb7365c18ff1de6440f80969dd9c81ab13b2388980a6a4a60cfc63cd7cc07f
Sep 15 16:27:29.323494 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:29.323430    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sqdn2"]
Sep 15 16:27:29.326254 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:27:29.326232    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad430be2_7c5f_4402_a5a7_c8b4359b8c02.slice/crio-12e647e7946732d64d2fc397e7cb3050c485feb4999f07cdabb521a320b3c57f WatchSource:0}: Error finding container 12e647e7946732d64d2fc397e7cb3050c485feb4999f07cdabb521a320b3c57f: Status 404 returned error can't find the container with id 12e647e7946732d64d2fc397e7cb3050c485feb4999f07cdabb521a320b3c57f
Sep 15 16:27:30.142631 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:30.142610    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:27:30.180483 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:30.180449    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-txzqk" event={"ID":"1d768ac0-1487-4850-bd9b-c66f73533e97","Type":"ContainerStarted","Data":"af9e2c03d728e92e7031f35f777e63cc49ba4aff4c7f56b09106c183fc889f51"}
Sep 15 16:27:30.184016 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:30.183986    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sqdn2" event={"ID":"ad430be2-7c5f-4402-a5a7-c8b4359b8c02","Type":"ContainerStarted","Data":"12e647e7946732d64d2fc397e7cb3050c485feb4999f07cdabb521a320b3c57f"}
Sep 15 16:27:30.186712 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:30.186686    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-27hdl" event={"ID":"abc7e760-ef1c-48bd-8d65-c5ad19f3f422","Type":"ContainerStarted","Data":"d6dcb8f097abce7c466e041a1d445c245eaa17b7e78f1797b2c324b2b1587334"}
Sep 15 16:27:30.186861 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:30.186745    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-27hdl" event={"ID":"abc7e760-ef1c-48bd-8d65-c5ad19f3f422","Type":"ContainerStarted","Data":"f08f323d925800f671d959cfec73160296195717a4eada0d1b439cb74258c4b0"}
Sep 15 16:27:30.186861 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:30.186795    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-27hdl" event={"ID":"abc7e760-ef1c-48bd-8d65-c5ad19f3f422","Type":"ContainerStarted","Data":"37eb7365c18ff1de6440f80969dd9c81ab13b2388980a6a4a60cfc63cd7cc07f"}
Sep 15 16:27:31.194541 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:31.193771    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-txzqk" event={"ID":"1d768ac0-1487-4850-bd9b-c66f73533e97","Type":"ContainerStarted","Data":"d1dd5017fc190f8f6ed30b7f011939d8a050e9a2b0f9826fd721c9689f5029f7"}
Sep 15 16:27:31.216282 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:31.216228    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-txzqk" podStartSLOduration=67.215682448 podStartE2EDuration="1m8.216206945s" totalImagesPullingTime="1.000524497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 16:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:27:31.215400045 +0000 UTC m=+67.845054398" watchObservedRunningTime="2026-09-15 16:27:31.216206945 +0000 UTC m=+67.845861299"
Sep 15 16:27:34.207320 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:34.207286    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9hbcw" event={"ID":"d1e544d2-21e5-4abd-82d9-33a82470f13c","Type":"ContainerStarted","Data":"be0584fe6caba9b5189d04ccd940b72cffe38b541de1248d1a410594213b2cc8"}
Sep 15 16:27:34.208879 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:34.208856    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sqdn2" event={"ID":"ad430be2-7c5f-4402-a5a7-c8b4359b8c02","Type":"ContainerStarted","Data":"791bc759de4d7381357edba11f25c989cf5b3fc5ee7bdc3bbee2572642ffc162"}
Sep 15 16:27:34.209003 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:34.208961    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:27:34.211166 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:34.211137    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-27hdl" event={"ID":"abc7e760-ef1c-48bd-8d65-c5ad19f3f422","Type":"ContainerStarted","Data":"a3ec79ba768640cfa365502f140a3df530ee363a3f59b30a35bfbd2b2967a712"}
Sep 15 16:27:34.225296 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:34.225252    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9hbcw" podStartSLOduration=66.225643856 podStartE2EDuration="1m11.225239375s" totalImagesPullingTime="4.999595519s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 16:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:27:34.224232115 +0000 UTC m=+70.853886465" watchObservedRunningTime="2026-09-15 16:27:34.225239375 +0000 UTC m=+70.854893728"
Sep 15 16:27:34.246092 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:34.246053    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-sqdn2" podStartSLOduration=66.532603926 podStartE2EDuration="1m11.246040143s" totalImagesPullingTime="4.713436217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 16:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:27:34.245328634 +0000 UTC m=+70.874982986" watchObservedRunningTime="2026-09-15 16:27:34.246040143 +0000 UTC m=+70.875694494"
Sep 15 16:27:34.264331 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:34.264242    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-27hdl" podStartSLOduration=33.562637404 podStartE2EDuration="38.264225813s" totalImagesPullingTime="4.701588409s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 16:26:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:27:34.262592001 +0000 UTC m=+70.892246352" watchObservedRunningTime="2026-09-15 16:27:34.264225813 +0000 UTC m=+70.893880165"
Sep 15 16:27:42.157277 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:42.157253    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8565fb9966-gk2rg"
Sep 15 16:27:45.154251 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:45.154113    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-8c846c645-h9j62" podUID="563c6d5b-c270-41de-bed4-b2aeb8fe8f42" containerName="registry" containerID="cri-o://9e81120a09b5c95595aef12003136ba87f07d224f361b6a246b96333a251ff59" gracePeriod=30
Sep 15 16:27:45.390123 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:45.390100    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:27:45.456682 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:45.456660    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-ca-trust-extracted\") pod \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") "
Sep 15 16:27:45.456807 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:45.456697    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-registry-certificates\") pod \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") "
Sep 15 16:27:45.456807 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:45.456723    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-trusted-ca\") pod \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") "
Sep 15 16:27:45.456807 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:45.456750    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-registry-tls\") pod \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") "
Sep 15 16:27:45.456807 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:45.456768    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-installation-pull-secrets\") pod \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") "
Sep 15 16:27:45.457020 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:45.456824    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-bound-sa-token\") pod \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") "
Sep 15 16:27:45.457020 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:45.456852    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ct8fd\" (UniqueName: \"kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-kube-api-access-ct8fd\") pod \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") "
Sep 15 16:27:45.457020 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:45.456880    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-image-registry-private-configuration\") pod \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\" (UID: \"563c6d5b-c270-41de-bed4-b2aeb8fe8f42\") "
Sep 15 16:27:45.457177 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:45.457146    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "563c6d5b-c270-41de-bed4-b2aeb8fe8f42" (UID: "563c6d5b-c270-41de-bed4-b2aeb8fe8f42"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:27:45.457563 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:45.457523    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "563c6d5b-c270-41de-bed4-b2aeb8fe8f42" (UID: "563c6d5b-c270-41de-bed4-b2aeb8fe8f42"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:27:45.459673 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:45.459647    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "563c6d5b-c270-41de-bed4-b2aeb8fe8f42" (UID: "563c6d5b-c270-41de-bed4-b2aeb8fe8f42"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 16:27:45.459962 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:45.459930    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "563c6d5b-c270-41de-bed4-b2aeb8fe8f42" (UID: "563c6d5b-c270-41de-bed4-b2aeb8fe8f42"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 16:27:45.459962 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:45.459939    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "563c6d5b-c270-41de-bed4-b2aeb8fe8f42" (UID: "563c6d5b-c270-41de-bed4-b2aeb8fe8f42"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 16:27:45.460063 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:45.459996    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "563c6d5b-c270-41de-bed4-b2aeb8fe8f42" (UID: "563c6d5b-c270-41de-bed4-b2aeb8fe8f42"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 16:27:45.460116 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:45.460089    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-kube-api-access-ct8fd" (OuterVolumeSpecName: "kube-api-access-ct8fd") pod "563c6d5b-c270-41de-bed4-b2aeb8fe8f42" (UID: "563c6d5b-c270-41de-bed4-b2aeb8fe8f42"). InnerVolumeSpecName "kube-api-access-ct8fd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 16:27:45.472930 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:45.472904    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "563c6d5b-c270-41de-bed4-b2aeb8fe8f42" (UID: "563c6d5b-c270-41de-bed4-b2aeb8fe8f42"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 16:27:45.557433 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:45.557410    2579 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-image-registry-private-configuration\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:27:45.557433 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:45.557432    2579 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-ca-trust-extracted\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:27:45.557568 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:45.557446    2579 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-registry-certificates\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:27:45.557568 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:45.557459    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-trusted-ca\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:27:45.557568 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:45.557471    2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-registry-tls\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:27:45.557568 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:45.557483    2579 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-installation-pull-secrets\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:27:45.557568 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:45.557495    2579 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-bound-sa-token\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:27:45.557568 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:45.557508    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ct8fd\" (UniqueName: \"kubernetes.io/projected/563c6d5b-c270-41de-bed4-b2aeb8fe8f42-kube-api-access-ct8fd\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:27:46.248530 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:46.248497    2579 generic.go:356] "Generic (PLEG): container finished" podID="563c6d5b-c270-41de-bed4-b2aeb8fe8f42" containerID="9e81120a09b5c95595aef12003136ba87f07d224f361b6a246b96333a251ff59" exitCode=0
Sep 15 16:27:46.248880 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:46.248534    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8c846c645-h9j62" event={"ID":"563c6d5b-c270-41de-bed4-b2aeb8fe8f42","Type":"ContainerDied","Data":"9e81120a09b5c95595aef12003136ba87f07d224f361b6a246b96333a251ff59"}
Sep 15 16:27:46.248880 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:46.248569    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8c846c645-h9j62"
Sep 15 16:27:46.248880 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:46.248591    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8c846c645-h9j62" event={"ID":"563c6d5b-c270-41de-bed4-b2aeb8fe8f42","Type":"ContainerDied","Data":"21819786c57d326bbf1d876996597f9dfa3058b3ca4a0fa82f02eb439f11dd37"}
Sep 15 16:27:46.248880 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:46.248622    2579 scope.go:117] "RemoveContainer" containerID="9e81120a09b5c95595aef12003136ba87f07d224f361b6a246b96333a251ff59"
Sep 15 16:27:46.256208 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:46.256189    2579 scope.go:117] "RemoveContainer" containerID="9e81120a09b5c95595aef12003136ba87f07d224f361b6a246b96333a251ff59"
Sep 15 16:27:46.256498 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:27:46.256474    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e81120a09b5c95595aef12003136ba87f07d224f361b6a246b96333a251ff59\": container with ID starting with 9e81120a09b5c95595aef12003136ba87f07d224f361b6a246b96333a251ff59 not found: ID does not exist" containerID="9e81120a09b5c95595aef12003136ba87f07d224f361b6a246b96333a251ff59"
Sep 15 16:27:46.256567 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:46.256510    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e81120a09b5c95595aef12003136ba87f07d224f361b6a246b96333a251ff59"} err="failed to get container status \"9e81120a09b5c95595aef12003136ba87f07d224f361b6a246b96333a251ff59\": rpc error: code = NotFound desc = could not find container \"9e81120a09b5c95595aef12003136ba87f07d224f361b6a246b96333a251ff59\": container with ID starting with 9e81120a09b5c95595aef12003136ba87f07d224f361b6a246b96333a251ff59 not found: ID does not exist"
Sep 15 16:27:46.284032 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:46.284006    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8c846c645-h9j62"]
Sep 15 16:27:46.296177 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:46.296155    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-8c846c645-h9j62"]
Sep 15 16:27:47.906471 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:47.906446    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="563c6d5b-c270-41de-bed4-b2aeb8fe8f42" path="/var/lib/kubelet/pods/563c6d5b-c270-41de-bed4-b2aeb8fe8f42/volumes"
Sep 15 16:27:54.179998 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.179962    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-g52bv"]
Sep 15 16:27:54.180386 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.180248    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="563c6d5b-c270-41de-bed4-b2aeb8fe8f42" containerName="registry"
Sep 15 16:27:54.180386 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.180261    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="563c6d5b-c270-41de-bed4-b2aeb8fe8f42" containerName="registry"
Sep 15 16:27:54.180386 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.180327    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="563c6d5b-c270-41de-bed4-b2aeb8fe8f42" containerName="registry"
Sep 15 16:27:54.185848 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.185822    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-g52bv"
Sep 15 16:27:54.188312 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.188283    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 15 16:27:54.189396 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.189321    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 15 16:27:54.189681 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.189662    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 15 16:27:54.189847 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.189817    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 15 16:27:54.189949 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.189871    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-gl97p\""
Sep 15 16:27:54.192386 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.192372    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 15 16:27:54.197589 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.197565    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-g52bv"]
Sep 15 16:27:54.208362 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.208334    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj5rd\" (UniqueName: \"kubernetes.io/projected/37670208-8bed-4d26-b9c2-069b51c3db6d-kube-api-access-wj5rd\") pod \"openshift-state-metrics-67dcf8c979-g52bv\" (UID: \"37670208-8bed-4d26-b9c2-069b51c3db6d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-g52bv"
Sep 15 16:27:54.208438 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.208375    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/37670208-8bed-4d26-b9c2-069b51c3db6d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-g52bv\" (UID: \"37670208-8bed-4d26-b9c2-069b51c3db6d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-g52bv"
Sep 15 16:27:54.208438 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.208423    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/37670208-8bed-4d26-b9c2-069b51c3db6d-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-g52bv\" (UID: \"37670208-8bed-4d26-b9c2-069b51c3db6d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-g52bv"
Sep 15 16:27:54.208522 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.208447    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/37670208-8bed-4d26-b9c2-069b51c3db6d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-g52bv\" (UID: \"37670208-8bed-4d26-b9c2-069b51c3db6d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-g52bv"
Sep 15 16:27:54.223960 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.223935    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-d2kqd"]
Sep 15 16:27:54.227670 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.227653    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d2kqd"
Sep 15 16:27:54.231708 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.231691    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 15 16:27:54.231959 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.231942    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 15 16:27:54.232013 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.231986    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 15 16:27:54.232239 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.232219    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jpbg9\""
Sep 15 16:27:54.309557 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.309538    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c7b5b20f-7476-4856-8cad-870d59f56499-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d2kqd\" (UID: \"c7b5b20f-7476-4856-8cad-870d59f56499\") " pod="openshift-monitoring/node-exporter-d2kqd"
Sep 15 16:27:54.309646 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.309579    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c7b5b20f-7476-4856-8cad-870d59f56499-node-exporter-tls\") pod \"node-exporter-d2kqd\" (UID: \"c7b5b20f-7476-4856-8cad-870d59f56499\") " pod="openshift-monitoring/node-exporter-d2kqd"
Sep 15 16:27:54.309646 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.309607    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c7b5b20f-7476-4856-8cad-870d59f56499-root\") pod \"node-exporter-d2kqd\" (UID: \"c7b5b20f-7476-4856-8cad-870d59f56499\") " pod="openshift-monitoring/node-exporter-d2kqd"
Sep 15 16:27:54.309646 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.309631    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7b5b20f-7476-4856-8cad-870d59f56499-sys\") pod \"node-exporter-d2kqd\" (UID: \"c7b5b20f-7476-4856-8cad-870d59f56499\") " pod="openshift-monitoring/node-exporter-d2kqd"
Sep 15 16:27:54.309803 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.309657    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c7b5b20f-7476-4856-8cad-870d59f56499-node-exporter-wtmp\") pod \"node-exporter-d2kqd\" (UID: \"c7b5b20f-7476-4856-8cad-870d59f56499\") " pod="openshift-monitoring/node-exporter-d2kqd"
Sep 15 16:27:54.309803 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.309680    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c7b5b20f-7476-4856-8cad-870d59f56499-node-exporter-textfile\") pod \"node-exporter-d2kqd\" (UID: \"c7b5b20f-7476-4856-8cad-870d59f56499\") " pod="openshift-monitoring/node-exporter-d2kqd"
Sep 15 16:27:54.309803 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.309708    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c7b5b20f-7476-4856-8cad-870d59f56499-node-exporter-accelerators-collector-config\") pod \"node-exporter-d2kqd\" (UID: \"c7b5b20f-7476-4856-8cad-870d59f56499\") " pod="openshift-monitoring/node-exporter-d2kqd"
Sep 15 16:27:54.309915 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.309808    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj5rd\" (UniqueName: \"kubernetes.io/projected/37670208-8bed-4d26-b9c2-069b51c3db6d-kube-api-access-wj5rd\") pod \"openshift-state-metrics-67dcf8c979-g52bv\" (UID: \"37670208-8bed-4d26-b9c2-069b51c3db6d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-g52bv"
Sep 15 16:27:54.309915 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.309844    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/37670208-8bed-4d26-b9c2-069b51c3db6d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-g52bv\" (UID: \"37670208-8bed-4d26-b9c2-069b51c3db6d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-g52bv"
Sep 15 16:27:54.309915 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.309875    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/37670208-8bed-4d26-b9c2-069b51c3db6d-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-g52bv\" (UID: \"37670208-8bed-4d26-b9c2-069b51c3db6d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-g52bv"
Sep 15 16:27:54.309915 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.309900    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mdhb\" (UniqueName: \"kubernetes.io/projected/c7b5b20f-7476-4856-8cad-870d59f56499-kube-api-access-7mdhb\") pod \"node-exporter-d2kqd\" (UID: \"c7b5b20f-7476-4856-8cad-870d59f56499\") " pod="openshift-monitoring/node-exporter-d2kqd"
Sep 15 16:27:54.310097 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.309927    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/37670208-8bed-4d26-b9c2-069b51c3db6d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-g52bv\" (UID: \"37670208-8bed-4d26-b9c2-069b51c3db6d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-g52bv"
Sep 15 16:27:54.310097 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.310060    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7b5b20f-7476-4856-8cad-870d59f56499-metrics-client-ca\") pod \"node-exporter-d2kqd\" (UID: \"c7b5b20f-7476-4856-8cad-870d59f56499\") " pod="openshift-monitoring/node-exporter-d2kqd"
Sep 15 16:27:54.310097 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:27:54.310080    2579 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 15 16:27:54.310237 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:27:54.310131    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37670208-8bed-4d26-b9c2-069b51c3db6d-openshift-state-metrics-tls podName:37670208-8bed-4d26-b9c2-069b51c3db6d nodeName:}" failed. No retries permitted until 2026-09-15 16:27:54.810112949 +0000 UTC m=+91.439767285 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/37670208-8bed-4d26-b9c2-069b51c3db6d-openshift-state-metrics-tls") pod "openshift-state-metrics-67dcf8c979-g52bv" (UID: "37670208-8bed-4d26-b9c2-069b51c3db6d") : secret "openshift-state-metrics-tls" not found
Sep 15 16:27:54.310615 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.310588    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/37670208-8bed-4d26-b9c2-069b51c3db6d-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-g52bv\" (UID: \"37670208-8bed-4d26-b9c2-069b51c3db6d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-g52bv"
Sep 15 16:27:54.312494 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.312466    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/37670208-8bed-4d26-b9c2-069b51c3db6d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-g52bv\" (UID: \"37670208-8bed-4d26-b9c2-069b51c3db6d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-g52bv"
Sep 15 16:27:54.325857 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.325831    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj5rd\" (UniqueName: \"kubernetes.io/projected/37670208-8bed-4d26-b9c2-069b51c3db6d-kube-api-access-wj5rd\") pod \"openshift-state-metrics-67dcf8c979-g52bv\" (UID: \"37670208-8bed-4d26-b9c2-069b51c3db6d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-g52bv"
Sep 15 16:27:54.410332 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.410313    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7b5b20f-7476-4856-8cad-870d59f56499-sys\") pod \"node-exporter-d2kqd\" (UID: \"c7b5b20f-7476-4856-8cad-870d59f56499\") " pod="openshift-monitoring/node-exporter-d2kqd"
Sep 15 16:27:54.410428 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.410340    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c7b5b20f-7476-4856-8cad-870d59f56499-node-exporter-wtmp\") pod \"node-exporter-d2kqd\" (UID: \"c7b5b20f-7476-4856-8cad-870d59f56499\") " pod="openshift-monitoring/node-exporter-d2kqd"
Sep 15 16:27:54.410428 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.410356    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c7b5b20f-7476-4856-8cad-870d59f56499-node-exporter-textfile\") pod \"node-exporter-d2kqd\" (UID: \"c7b5b20f-7476-4856-8cad-870d59f56499\") " pod="openshift-monitoring/node-exporter-d2kqd"
Sep 15 16:27:54.410428 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.410373    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c7b5b20f-7476-4856-8cad-870d59f56499-node-exporter-accelerators-collector-config\") pod \"node-exporter-d2kqd\" (UID: \"c7b5b20f-7476-4856-8cad-870d59f56499\") " pod="openshift-monitoring/node-exporter-d2kqd"
Sep 15 16:27:54.410428 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.410412    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mdhb\" (UniqueName: \"kubernetes.io/projected/c7b5b20f-7476-4856-8cad-870d59f56499-kube-api-access-7mdhb\") pod \"node-exporter-d2kqd\" (UID: \"c7b5b20f-7476-4856-8cad-870d59f56499\") " pod="openshift-monitoring/node-exporter-d2kqd"
Sep 15 16:27:54.410600 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.410448    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7b5b20f-7476-4856-8cad-870d59f56499-metrics-client-ca\") pod \"node-exporter-d2kqd\" (UID: \"c7b5b20f-7476-4856-8cad-870d59f56499\") " pod="openshift-monitoring/node-exporter-d2kqd"
Sep 15 16:27:54.410600 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.410456    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7b5b20f-7476-4856-8cad-870d59f56499-sys\") pod \"node-exporter-d2kqd\" (UID: \"c7b5b20f-7476-4856-8cad-870d59f56499\") " pod="openshift-monitoring/node-exporter-d2kqd"
Sep 15 16:27:54.410600 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.410480    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c7b5b20f-7476-4856-8cad-870d59f56499-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d2kqd\" (UID: \"c7b5b20f-7476-4856-8cad-870d59f56499\") " pod="openshift-monitoring/node-exporter-d2kqd"
Sep 15 16:27:54.410600 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.410495    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c7b5b20f-7476-4856-8cad-870d59f56499-node-exporter-wtmp\") pod \"node-exporter-d2kqd\" (UID: \"c7b5b20f-7476-4856-8cad-870d59f56499\") " pod="openshift-monitoring/node-exporter-d2kqd"
Sep 15 16:27:54.410600 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.410511    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c7b5b20f-7476-4856-8cad-870d59f56499-node-exporter-tls\") pod \"node-exporter-d2kqd\" (UID: \"c7b5b20f-7476-4856-8cad-870d59f56499\") " pod="openshift-monitoring/node-exporter-d2kqd"
Sep 15 16:27:54.410600 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.410549    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c7b5b20f-7476-4856-8cad-870d59f56499-root\") pod \"node-exporter-d2kqd\" (UID: \"c7b5b20f-7476-4856-8cad-870d59f56499\") " pod="openshift-monitoring/node-exporter-d2kqd"
Sep 15 16:27:54.410600 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:27:54.410590    2579 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 15 16:27:54.410934 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.410605    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c7b5b20f-7476-4856-8cad-870d59f56499-root\") pod \"node-exporter-d2kqd\" (UID: \"c7b5b20f-7476-4856-8cad-870d59f56499\") " pod="openshift-monitoring/node-exporter-d2kqd"
Sep 15 16:27:54.410934 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:27:54.410638    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7b5b20f-7476-4856-8cad-870d59f56499-node-exporter-tls podName:c7b5b20f-7476-4856-8cad-870d59f56499 nodeName:}" failed. No retries permitted until 2026-09-15 16:27:54.910622328 +0000 UTC m=+91.540276658 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/c7b5b20f-7476-4856-8cad-870d59f56499-node-exporter-tls") pod "node-exporter-d2kqd" (UID: "c7b5b20f-7476-4856-8cad-870d59f56499") : secret "node-exporter-tls" not found
Sep 15 16:27:54.410934 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.410649    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c7b5b20f-7476-4856-8cad-870d59f56499-node-exporter-textfile\") pod \"node-exporter-d2kqd\" (UID: \"c7b5b20f-7476-4856-8cad-870d59f56499\") " pod="openshift-monitoring/node-exporter-d2kqd"
Sep 15 16:27:54.411073 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.411042    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c7b5b20f-7476-4856-8cad-870d59f56499-node-exporter-accelerators-collector-config\") pod \"node-exporter-d2kqd\" (UID: \"c7b5b20f-7476-4856-8cad-870d59f56499\") " pod="openshift-monitoring/node-exporter-d2kqd"
Sep 15 16:27:54.411116 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.411074    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7b5b20f-7476-4856-8cad-870d59f56499-metrics-client-ca\") pod \"node-exporter-d2kqd\" (UID: \"c7b5b20f-7476-4856-8cad-870d59f56499\") " pod="openshift-monitoring/node-exporter-d2kqd"
Sep 15 16:27:54.412652 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.412625    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c7b5b20f-7476-4856-8cad-870d59f56499-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d2kqd\" (UID: \"c7b5b20f-7476-4856-8cad-870d59f56499\") " pod="openshift-monitoring/node-exporter-d2kqd"
Sep 15 16:27:54.418743 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.418726    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mdhb\" (UniqueName: \"kubernetes.io/projected/c7b5b20f-7476-4856-8cad-870d59f56499-kube-api-access-7mdhb\") pod \"node-exporter-d2kqd\" (UID: \"c7b5b20f-7476-4856-8cad-870d59f56499\") " pod="openshift-monitoring/node-exporter-d2kqd"
Sep 15 16:27:54.813663 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.813634    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/37670208-8bed-4d26-b9c2-069b51c3db6d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-g52bv\" (UID: \"37670208-8bed-4d26-b9c2-069b51c3db6d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-g52bv"
Sep 15 16:27:54.815913 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.815885    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/37670208-8bed-4d26-b9c2-069b51c3db6d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-g52bv\" (UID: \"37670208-8bed-4d26-b9c2-069b51c3db6d\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-g52bv"
Sep 15 16:27:54.914455 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.914414    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c7b5b20f-7476-4856-8cad-870d59f56499-node-exporter-tls\") pod \"node-exporter-d2kqd\" (UID: \"c7b5b20f-7476-4856-8cad-870d59f56499\") " pod="openshift-monitoring/node-exporter-d2kqd"
Sep 15 16:27:54.916452 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:54.916436    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c7b5b20f-7476-4856-8cad-870d59f56499-node-exporter-tls\") pod \"node-exporter-d2kqd\" (UID: \"c7b5b20f-7476-4856-8cad-870d59f56499\") " pod="openshift-monitoring/node-exporter-d2kqd"
Sep 15 16:27:55.096805 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.096739    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-g52bv"
Sep 15 16:27:55.135897 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.135871    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d2kqd"
Sep 15 16:27:55.147618 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:27:55.147588    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7b5b20f_7476_4856_8cad_870d59f56499.slice/crio-67091a57edec803c161f428ffd6adefee3bb2d2ba2c1f62d232893bf719737d6 WatchSource:0}: Error finding container 67091a57edec803c161f428ffd6adefee3bb2d2ba2c1f62d232893bf719737d6: Status 404 returned error can't find the container with id 67091a57edec803c161f428ffd6adefee3bb2d2ba2c1f62d232893bf719737d6
Sep 15 16:27:55.281603 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.281563    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d2kqd" event={"ID":"c7b5b20f-7476-4856-8cad-870d59f56499","Type":"ContainerStarted","Data":"67091a57edec803c161f428ffd6adefee3bb2d2ba2c1f62d232893bf719737d6"}
Sep 15 16:27:55.319577 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.319544    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 15 16:27:55.324339 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.324319    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.327707 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.327687    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 15 16:27:55.327707 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.327701    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 15 16:27:55.328058 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.328039    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-pbbkh\""
Sep 15 16:27:55.328133 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.328094    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 15 16:27:55.328190 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.328140    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 15 16:27:55.328190 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.328157    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 15 16:27:55.328190 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.328145    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 15 16:27:55.328353 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.328265    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 15 16:27:55.328535 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.328521    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 15 16:27:55.328943 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.328927    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 15 16:27:55.342274 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.342252    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 15 16:27:55.382687 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.382674    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-g52bv"]
Sep 15 16:27:55.384969 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:27:55.384950    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37670208_8bed_4d26_b9c2_069b51c3db6d.slice/crio-b9042ab9753ab4b71a6bb5e9c24ee1301d50570250c647ab41a2b0a4637cadd6 WatchSource:0}: Error finding container b9042ab9753ab4b71a6bb5e9c24ee1301d50570250c647ab41a2b0a4637cadd6: Status 404 returned error can't find the container with id b9042ab9753ab4b71a6bb5e9c24ee1301d50570250c647ab41a2b0a4637cadd6
Sep 15 16:27:55.419647 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.419622    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.419738 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.419666    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.419738 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.419705    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.419864 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.419772    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-web-config\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.419864 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.419826    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.419864 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.419849    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.419986 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.419885    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.419986 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.419924    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-config-volume\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.419986 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.419941    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.419986 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.419970    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-config-out\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.420103 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.420007    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rj6lj\" (UniqueName: \"kubernetes.io/projected/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-kube-api-access-rj6lj\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.420103 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.420059    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.420103 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.420088    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.520930 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.520907    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.521039 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.520937    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.521039 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.520954    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.521039 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.520971    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-web-config\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.521039 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.520990    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.521233 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.521165    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.521233 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.521203    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.521233 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.521231    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-config-volume\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.521389 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.521251    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.521389 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.521290    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-config-out\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.521389 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.521312    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rj6lj\" (UniqueName: \"kubernetes.io/projected/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-kube-api-access-rj6lj\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.521389 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.521340    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.521389 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.521368    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.521676 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.521657    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.522252 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:27:55.522230    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-alertmanager-trusted-ca-bundle podName:15efc309-7d8f-4b51-92e2-3c8e09ca50b0 nodeName:}" failed. No retries permitted until 2026-09-15 16:27:56.022209559 +0000 UTC m=+92.651863909 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "15efc309-7d8f-4b51-92e2-3c8e09ca50b0") : configmap references non-existent config key: ca-bundle.crt
Sep 15 16:27:55.522654 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.522583    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.523998 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.523927    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.523998 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.523959    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.524194 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.524170    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.524603 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.524586    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-config-out\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.525440 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.525392    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.526620 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.526602    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.526620 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.526613    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-web-config\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.527102 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.527083    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-config-volume\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.527148 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.527134    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:55.537281 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:55.537262    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rj6lj\" (UniqueName: \"kubernetes.io/projected/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-kube-api-access-rj6lj\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:56.024757 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:56.024721    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:56.025667 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:56.025645    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:56.232485 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:56.232459    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:27:56.298659 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:56.298047    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-g52bv" event={"ID":"37670208-8bed-4d26-b9c2-069b51c3db6d","Type":"ContainerStarted","Data":"309ce722ea1872094a0efcad985b526b3952e039cb4cdd5fc28056a7017703b7"}
Sep 15 16:27:56.298659 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:56.298119    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-g52bv" event={"ID":"37670208-8bed-4d26-b9c2-069b51c3db6d","Type":"ContainerStarted","Data":"2cab7e52a715c94906b30c2926b1c4daa5afebfdd068ecd9626db71eb200a5a1"}
Sep 15 16:27:56.298659 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:56.298149    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-g52bv" event={"ID":"37670208-8bed-4d26-b9c2-069b51c3db6d","Type":"ContainerStarted","Data":"b9042ab9753ab4b71a6bb5e9c24ee1301d50570250c647ab41a2b0a4637cadd6"}
Sep 15 16:27:56.301275 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:56.301226    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d2kqd" event={"ID":"c7b5b20f-7476-4856-8cad-870d59f56499","Type":"ContainerStarted","Data":"70727f1c5e7d3da8db01e231de62ed5bfae12802f083b561ec1136c02b1fdb43"}
Sep 15 16:27:56.523893 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:56.523853    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 15 16:27:56.695202 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:27:56.695142    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15efc309_7d8f_4b51_92e2_3c8e09ca50b0.slice/crio-203e0aa6a06bd551b392c0fe2277f9a4b7c59a4611bb7dab99678e3401c90eb7 WatchSource:0}: Error finding container 203e0aa6a06bd551b392c0fe2277f9a4b7c59a4611bb7dab99678e3401c90eb7: Status 404 returned error can't find the container with id 203e0aa6a06bd551b392c0fe2277f9a4b7c59a4611bb7dab99678e3401c90eb7
Sep 15 16:27:57.200858 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.200825    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-759d5984cf-cx47r"]
Sep 15 16:27:57.204793 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.204762    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r"
Sep 15 16:27:57.207382 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.207354    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-bhzdm\""
Sep 15 16:27:57.207382 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.207364    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 15 16:27:57.207560 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.207485    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 15 16:27:57.207697 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.207683    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-7vsgl5bdeb4v9\""
Sep 15 16:27:57.207772 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.207757    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 15 16:27:57.208195 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.208179    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 15 16:27:57.208268 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.208195    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 15 16:27:57.218258 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.218233    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-759d5984cf-cx47r"]
Sep 15 16:27:57.234949 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.234926    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/763cda34-2afd-45fd-8b1d-c23899dfb0b2-secret-thanos-querier-tls\") pod \"thanos-querier-759d5984cf-cx47r\" (UID: \"763cda34-2afd-45fd-8b1d-c23899dfb0b2\") " pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r"
Sep 15 16:27:57.235048 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.234962    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/763cda34-2afd-45fd-8b1d-c23899dfb0b2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-759d5984cf-cx47r\" (UID: \"763cda34-2afd-45fd-8b1d-c23899dfb0b2\") " pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r"
Sep 15 16:27:57.235048 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.234990    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5ndg\" (UniqueName: \"kubernetes.io/projected/763cda34-2afd-45fd-8b1d-c23899dfb0b2-kube-api-access-l5ndg\") pod \"thanos-querier-759d5984cf-cx47r\" (UID: \"763cda34-2afd-45fd-8b1d-c23899dfb0b2\") " pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r"
Sep 15 16:27:57.235165 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.235065    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/763cda34-2afd-45fd-8b1d-c23899dfb0b2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-759d5984cf-cx47r\" (UID: \"763cda34-2afd-45fd-8b1d-c23899dfb0b2\") " pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r"
Sep 15 16:27:57.235165 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.235093    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/763cda34-2afd-45fd-8b1d-c23899dfb0b2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-759d5984cf-cx47r\" (UID: \"763cda34-2afd-45fd-8b1d-c23899dfb0b2\") " pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r"
Sep 15 16:27:57.235165 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.235116    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/763cda34-2afd-45fd-8b1d-c23899dfb0b2-secret-grpc-tls\") pod \"thanos-querier-759d5984cf-cx47r\" (UID: \"763cda34-2afd-45fd-8b1d-c23899dfb0b2\") " pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r"
Sep 15 16:27:57.235266 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.235211    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/763cda34-2afd-45fd-8b1d-c23899dfb0b2-metrics-client-ca\") pod \"thanos-querier-759d5984cf-cx47r\" (UID: \"763cda34-2afd-45fd-8b1d-c23899dfb0b2\") " pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r"
Sep 15 16:27:57.235307 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.235268    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/763cda34-2afd-45fd-8b1d-c23899dfb0b2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-759d5984cf-cx47r\" (UID: \"763cda34-2afd-45fd-8b1d-c23899dfb0b2\") " pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r"
Sep 15 16:27:57.307489 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.307456    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"15efc309-7d8f-4b51-92e2-3c8e09ca50b0","Type":"ContainerStarted","Data":"203e0aa6a06bd551b392c0fe2277f9a4b7c59a4611bb7dab99678e3401c90eb7"}
Sep 15 16:27:57.309696 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.309674    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-g52bv" event={"ID":"37670208-8bed-4d26-b9c2-069b51c3db6d","Type":"ContainerStarted","Data":"9c3aec70adefdf065c75805f79f209b8d0876c201e54e8df855d517c504ecfbd"}
Sep 15 16:27:57.311859 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.311838    2579 generic.go:356] "Generic (PLEG): container finished" podID="c7b5b20f-7476-4856-8cad-870d59f56499" containerID="70727f1c5e7d3da8db01e231de62ed5bfae12802f083b561ec1136c02b1fdb43" exitCode=0
Sep 15 16:27:57.311939 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.311883    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d2kqd" event={"ID":"c7b5b20f-7476-4856-8cad-870d59f56499","Type":"ContainerDied","Data":"70727f1c5e7d3da8db01e231de62ed5bfae12802f083b561ec1136c02b1fdb43"}
Sep 15 16:27:57.311939 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.311920    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d2kqd" event={"ID":"c7b5b20f-7476-4856-8cad-870d59f56499","Type":"ContainerStarted","Data":"aec4229bc772074eb502097c28e548f5158e4836ff128bcd79a5f16c2d12da82"}
Sep 15 16:27:57.312026 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.311942    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d2kqd" event={"ID":"c7b5b20f-7476-4856-8cad-870d59f56499","Type":"ContainerStarted","Data":"8c279e34a744cdf4330f491ba8f7b5dcecd0cfed1f5781668f65b2f118ee9d40"}
Sep 15 16:27:57.330219 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.330171    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-g52bv" podStartSLOduration=2.091643547 podStartE2EDuration="3.33015853s" totalImagesPullingTime="1.238514983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 16:27:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:27:57.328859373 +0000 UTC m=+93.958513813" watchObservedRunningTime="2026-09-15 16:27:57.33015853 +0000 UTC m=+93.959812891"
Sep 15 16:27:57.336145 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.336118    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/763cda34-2afd-45fd-8b1d-c23899dfb0b2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-759d5984cf-cx47r\" (UID: \"763cda34-2afd-45fd-8b1d-c23899dfb0b2\") " pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r"
Sep 15 16:27:57.336228 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.336162    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/763cda34-2afd-45fd-8b1d-c23899dfb0b2-secret-thanos-querier-tls\") pod \"thanos-querier-759d5984cf-cx47r\" (UID: \"763cda34-2afd-45fd-8b1d-c23899dfb0b2\") " pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r"
Sep 15 16:27:57.336228 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.336194    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/763cda34-2afd-45fd-8b1d-c23899dfb0b2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-759d5984cf-cx47r\" (UID: \"763cda34-2afd-45fd-8b1d-c23899dfb0b2\") " pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r"
Sep 15 16:27:57.336228 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.336223    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5ndg\" (UniqueName: \"kubernetes.io/projected/763cda34-2afd-45fd-8b1d-c23899dfb0b2-kube-api-access-l5ndg\") pod \"thanos-querier-759d5984cf-cx47r\" (UID: \"763cda34-2afd-45fd-8b1d-c23899dfb0b2\") " pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r"
Sep 15 16:27:57.336393 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.336345    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/763cda34-2afd-45fd-8b1d-c23899dfb0b2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-759d5984cf-cx47r\" (UID: \"763cda34-2afd-45fd-8b1d-c23899dfb0b2\") " pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r"
Sep 15 16:27:57.336444 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.336388    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/763cda34-2afd-45fd-8b1d-c23899dfb0b2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-759d5984cf-cx47r\" (UID: \"763cda34-2afd-45fd-8b1d-c23899dfb0b2\") " pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r"
Sep 15 16:27:57.336444 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.336422    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/763cda34-2afd-45fd-8b1d-c23899dfb0b2-secret-grpc-tls\") pod \"thanos-querier-759d5984cf-cx47r\" (UID: \"763cda34-2afd-45fd-8b1d-c23899dfb0b2\") " pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r"
Sep 15 16:27:57.336545 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.336473    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/763cda34-2afd-45fd-8b1d-c23899dfb0b2-metrics-client-ca\") pod \"thanos-querier-759d5984cf-cx47r\" (UID: \"763cda34-2afd-45fd-8b1d-c23899dfb0b2\") " pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r"
Sep 15 16:27:57.337704 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.337677    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/763cda34-2afd-45fd-8b1d-c23899dfb0b2-metrics-client-ca\") pod \"thanos-querier-759d5984cf-cx47r\" (UID: \"763cda34-2afd-45fd-8b1d-c23899dfb0b2\") " pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r"
Sep 15 16:27:57.339033 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.339004    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/763cda34-2afd-45fd-8b1d-c23899dfb0b2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-759d5984cf-cx47r\" (UID: \"763cda34-2afd-45fd-8b1d-c23899dfb0b2\") " pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r"
Sep 15 16:27:57.339365 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.339339    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/763cda34-2afd-45fd-8b1d-c23899dfb0b2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-759d5984cf-cx47r\" (UID: \"763cda34-2afd-45fd-8b1d-c23899dfb0b2\") " pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r"
Sep 15 16:27:57.339532 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.339514    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/763cda34-2afd-45fd-8b1d-c23899dfb0b2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-759d5984cf-cx47r\" (UID: \"763cda34-2afd-45fd-8b1d-c23899dfb0b2\") " pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r"
Sep 15 16:27:57.339643 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.339626    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/763cda34-2afd-45fd-8b1d-c23899dfb0b2-secret-thanos-querier-tls\") pod \"thanos-querier-759d5984cf-cx47r\" (UID: \"763cda34-2afd-45fd-8b1d-c23899dfb0b2\") " pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r"
Sep 15 16:27:57.339752 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.339733    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/763cda34-2afd-45fd-8b1d-c23899dfb0b2-secret-grpc-tls\") pod \"thanos-querier-759d5984cf-cx47r\" (UID: \"763cda34-2afd-45fd-8b1d-c23899dfb0b2\") " pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r"
Sep 15 16:27:57.339947 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.339929    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/763cda34-2afd-45fd-8b1d-c23899dfb0b2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-759d5984cf-cx47r\" (UID: \"763cda34-2afd-45fd-8b1d-c23899dfb0b2\") " pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r"
Sep 15 16:27:57.349548 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.349510    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-d2kqd" podStartSLOduration=2.275167167 podStartE2EDuration="3.349499604s" totalImagesPullingTime="999.173687ms" totalInitContainerRuntime="75.15875ms" isStatefulPod=true podCreationTimestamp="2026-09-15 16:27:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:27:57.348660866 +0000 UTC m=+93.978315224" watchObservedRunningTime="2026-09-15 16:27:57.349499604 +0000 UTC m=+93.979153990"
Sep 15 16:27:57.351659 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.351640    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5ndg\" (UniqueName: \"kubernetes.io/projected/763cda34-2afd-45fd-8b1d-c23899dfb0b2-kube-api-access-l5ndg\") pod \"thanos-querier-759d5984cf-cx47r\" (UID: \"763cda34-2afd-45fd-8b1d-c23899dfb0b2\") " pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r"
Sep 15 16:27:57.513989 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.513963    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r"
Sep 15 16:27:57.791016 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:57.790946    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-759d5984cf-cx47r"]
Sep 15 16:27:57.793621 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:27:57.793601    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod763cda34_2afd_45fd_8b1d_c23899dfb0b2.slice/crio-e599eef9fb932adc817fa0f99c9225f7f91ac41f0b0a37ca5536f6b8a1ec5dfc WatchSource:0}: Error finding container e599eef9fb932adc817fa0f99c9225f7f91ac41f0b0a37ca5536f6b8a1ec5dfc: Status 404 returned error can't find the container with id e599eef9fb932adc817fa0f99c9225f7f91ac41f0b0a37ca5536f6b8a1ec5dfc
Sep 15 16:27:58.319558 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:58.319524    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r" event={"ID":"763cda34-2afd-45fd-8b1d-c23899dfb0b2","Type":"ContainerStarted","Data":"e599eef9fb932adc817fa0f99c9225f7f91ac41f0b0a37ca5536f6b8a1ec5dfc"}
Sep 15 16:27:59.324999 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:59.324963    2579 generic.go:356] "Generic (PLEG): container finished" podID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerID="6a1a9295b654cb0764dbfa7ba10f810695adef741acada36b6aec515a290f381" exitCode=0
Sep 15 16:27:59.325377 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:27:59.325046    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"15efc309-7d8f-4b51-92e2-3c8e09ca50b0","Type":"ContainerDied","Data":"6a1a9295b654cb0764dbfa7ba10f810695adef741acada36b6aec515a290f381"}
Sep 15 16:28:00.331670 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:00.331633    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r" event={"ID":"763cda34-2afd-45fd-8b1d-c23899dfb0b2","Type":"ContainerStarted","Data":"bc7778211d4e49a254fd8a0f2b5939711ada2a217454590df6a96a3e262e62c0"}
Sep 15 16:28:00.332070 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:00.331715    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r" event={"ID":"763cda34-2afd-45fd-8b1d-c23899dfb0b2","Type":"ContainerStarted","Data":"74328932fd6541a61af80b16c36d5b75384732c6d84af3a884613b983b696ed5"}
Sep 15 16:28:00.332070 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:00.331757    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r" event={"ID":"763cda34-2afd-45fd-8b1d-c23899dfb0b2","Type":"ContainerStarted","Data":"d5b3941306dec34a15e433e6f1089859a8f66a95727247dc6307a72adfdad074"}
Sep 15 16:28:01.339525 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:01.339492    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"15efc309-7d8f-4b51-92e2-3c8e09ca50b0","Type":"ContainerStarted","Data":"a609ecbe95e9aba12953a63b964274cf60bf1978c9768c8aa483e30adcfd6adc"}
Sep 15 16:28:01.339889 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:01.339604    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"15efc309-7d8f-4b51-92e2-3c8e09ca50b0","Type":"ContainerStarted","Data":"259487950e76627062b673e90c62b2944ef97d5701a6322985f39c7dd5bf1541"}
Sep 15 16:28:01.339889 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:01.339646    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"15efc309-7d8f-4b51-92e2-3c8e09ca50b0","Type":"ContainerStarted","Data":"903e3c8a6d908cf8120b7933d600e17a097193917ed56932e208622d33d511d9"}
Sep 15 16:28:01.339889 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:01.339685    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"15efc309-7d8f-4b51-92e2-3c8e09ca50b0","Type":"ContainerStarted","Data":"c1de56608d4b0b4253d092c07ea108f05f67152444ffa9d7701f973185e91687"}
Sep 15 16:28:01.339889 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:01.339724    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"15efc309-7d8f-4b51-92e2-3c8e09ca50b0","Type":"ContainerStarted","Data":"cc69740c1f7dbc3d7b970647cf78a3b4baba256969cfa7560096298a87e18e56"}
Sep 15 16:28:01.343187 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:01.343160    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r" event={"ID":"763cda34-2afd-45fd-8b1d-c23899dfb0b2","Type":"ContainerStarted","Data":"75c60dd7b0fcf765386a73f289a2abb8c389784cb34d87b1fbe5a5c89aeee0e5"}
Sep 15 16:28:01.343267 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:01.343236    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r" event={"ID":"763cda34-2afd-45fd-8b1d-c23899dfb0b2","Type":"ContainerStarted","Data":"9e9c91b3fb4b0114fbead619eef8a7fe38ab098e11b4325e178810710b933ce6"}
Sep 15 16:28:01.343318 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:01.343278    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r" event={"ID":"763cda34-2afd-45fd-8b1d-c23899dfb0b2","Type":"ContainerStarted","Data":"41ce6f4540d2b19eb00e29b4f4df319acb6a69aeadac70f6cf038a25e244c131"}
Sep 15 16:28:01.343909 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:01.343849    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r"
Sep 15 16:28:01.368874 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:01.368821    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r" podStartSLOduration=1.307839043 podStartE2EDuration="4.368806722s" totalImagesPullingTime="3.060967679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 16:27:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:28:01.366334605 +0000 UTC m=+97.995988957" watchObservedRunningTime="2026-09-15 16:28:01.368806722 +0000 UTC m=+97.998461065"
Sep 15 16:28:02.349470 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.349435    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"15efc309-7d8f-4b51-92e2-3c8e09ca50b0","Type":"ContainerStarted","Data":"e6dd289119f786b25398b2122cf49df1bdd3bc10b6b40bd3262a9d9c7d480c2b"}
Sep 15 16:28:02.376544 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.376503    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.702229392 podStartE2EDuration="7.376489363s" totalImagesPullingTime="3.657553441s" totalInitContainerRuntime="16.70653ms" isStatefulPod=false podCreationTimestamp="2026-09-15 16:27:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:28:02.375459083 +0000 UTC m=+99.005113435" watchObservedRunningTime="2026-09-15 16:28:02.376489363 +0000 UTC m=+99.006143715"
Sep 15 16:28:02.447659 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.447635    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6847b7689-v74xn"]
Sep 15 16:28:02.453171 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.453156    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6847b7689-v74xn"
Sep 15 16:28:02.455562 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.455539    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 15 16:28:02.455868 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.455835    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 15 16:28:02.455868 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.455857    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 15 16:28:02.456121 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.456063    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 15 16:28:02.456192 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.456143    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 15 16:28:02.456192 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.456109    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4r7b8\""
Sep 15 16:28:02.456293 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.456208    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 15 16:28:02.456448 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.456434    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 15 16:28:02.461154 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.461137    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 15 16:28:02.469214 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.469194    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6847b7689-v74xn"]
Sep 15 16:28:02.481081 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.481062    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c691124b-f854-447e-893f-5db54c13a05c-service-ca\") pod \"console-6847b7689-v74xn\" (UID: \"c691124b-f854-447e-893f-5db54c13a05c\") " pod="openshift-console/console-6847b7689-v74xn"
Sep 15 16:28:02.481174 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.481086    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c691124b-f854-447e-893f-5db54c13a05c-trusted-ca-bundle\") pod \"console-6847b7689-v74xn\" (UID: \"c691124b-f854-447e-893f-5db54c13a05c\") " pod="openshift-console/console-6847b7689-v74xn"
Sep 15 16:28:02.481174 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.481104    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfpxg\" (UniqueName: \"kubernetes.io/projected/c691124b-f854-447e-893f-5db54c13a05c-kube-api-access-nfpxg\") pod \"console-6847b7689-v74xn\" (UID: \"c691124b-f854-447e-893f-5db54c13a05c\") " pod="openshift-console/console-6847b7689-v74xn"
Sep 15 16:28:02.481174 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.481122    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c691124b-f854-447e-893f-5db54c13a05c-console-serving-cert\") pod \"console-6847b7689-v74xn\" (UID: \"c691124b-f854-447e-893f-5db54c13a05c\") " pod="openshift-console/console-6847b7689-v74xn"
Sep 15 16:28:02.481269 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.481173    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c691124b-f854-447e-893f-5db54c13a05c-console-config\") pod \"console-6847b7689-v74xn\" (UID: \"c691124b-f854-447e-893f-5db54c13a05c\") " pod="openshift-console/console-6847b7689-v74xn"
Sep 15 16:28:02.481269 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.481246    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c691124b-f854-447e-893f-5db54c13a05c-console-oauth-config\") pod \"console-6847b7689-v74xn\" (UID: \"c691124b-f854-447e-893f-5db54c13a05c\") " pod="openshift-console/console-6847b7689-v74xn"
Sep 15 16:28:02.481328 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.481306    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c691124b-f854-447e-893f-5db54c13a05c-oauth-serving-cert\") pod \"console-6847b7689-v74xn\" (UID: \"c691124b-f854-447e-893f-5db54c13a05c\") " pod="openshift-console/console-6847b7689-v74xn"
Sep 15 16:28:02.582394 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.582370    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c691124b-f854-447e-893f-5db54c13a05c-console-oauth-config\") pod \"console-6847b7689-v74xn\" (UID: \"c691124b-f854-447e-893f-5db54c13a05c\") " pod="openshift-console/console-6847b7689-v74xn"
Sep 15 16:28:02.582472 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.582422    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c691124b-f854-447e-893f-5db54c13a05c-oauth-serving-cert\") pod \"console-6847b7689-v74xn\" (UID: \"c691124b-f854-447e-893f-5db54c13a05c\") " pod="openshift-console/console-6847b7689-v74xn"
Sep 15 16:28:02.582472 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.582447    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c691124b-f854-447e-893f-5db54c13a05c-service-ca\") pod \"console-6847b7689-v74xn\" (UID: \"c691124b-f854-447e-893f-5db54c13a05c\") " pod="openshift-console/console-6847b7689-v74xn"
Sep 15 16:28:02.582472 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.582461    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c691124b-f854-447e-893f-5db54c13a05c-trusted-ca-bundle\") pod \"console-6847b7689-v74xn\" (UID: \"c691124b-f854-447e-893f-5db54c13a05c\") " pod="openshift-console/console-6847b7689-v74xn"
Sep 15 16:28:02.582609 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.582477    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfpxg\" (UniqueName: \"kubernetes.io/projected/c691124b-f854-447e-893f-5db54c13a05c-kube-api-access-nfpxg\") pod \"console-6847b7689-v74xn\" (UID: \"c691124b-f854-447e-893f-5db54c13a05c\") " pod="openshift-console/console-6847b7689-v74xn"
Sep 15 16:28:02.582609 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.582495    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c691124b-f854-447e-893f-5db54c13a05c-console-serving-cert\") pod \"console-6847b7689-v74xn\" (UID: \"c691124b-f854-447e-893f-5db54c13a05c\") " pod="openshift-console/console-6847b7689-v74xn"
Sep 15 16:28:02.582609 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.582515    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c691124b-f854-447e-893f-5db54c13a05c-console-config\") pod \"console-6847b7689-v74xn\" (UID: \"c691124b-f854-447e-893f-5db54c13a05c\") " pod="openshift-console/console-6847b7689-v74xn"
Sep 15 16:28:02.583299 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.583269    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c691124b-f854-447e-893f-5db54c13a05c-console-config\") pod \"console-6847b7689-v74xn\" (UID: \"c691124b-f854-447e-893f-5db54c13a05c\") " pod="openshift-console/console-6847b7689-v74xn"
Sep 15 16:28:02.583386 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.583313    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c691124b-f854-447e-893f-5db54c13a05c-service-ca\") pod \"console-6847b7689-v74xn\" (UID: \"c691124b-f854-447e-893f-5db54c13a05c\") " pod="openshift-console/console-6847b7689-v74xn"
Sep 15 16:28:02.583429 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.583407    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c691124b-f854-447e-893f-5db54c13a05c-trusted-ca-bundle\") pod \"console-6847b7689-v74xn\" (UID: \"c691124b-f854-447e-893f-5db54c13a05c\") " pod="openshift-console/console-6847b7689-v74xn"
Sep 15 16:28:02.583502 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.583482    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c691124b-f854-447e-893f-5db54c13a05c-oauth-serving-cert\") pod \"console-6847b7689-v74xn\" (UID: \"c691124b-f854-447e-893f-5db54c13a05c\") " pod="openshift-console/console-6847b7689-v74xn"
Sep 15 16:28:02.584896 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.584874    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c691124b-f854-447e-893f-5db54c13a05c-console-oauth-config\") pod \"console-6847b7689-v74xn\" (UID: \"c691124b-f854-447e-893f-5db54c13a05c\") " pod="openshift-console/console-6847b7689-v74xn"
Sep 15 16:28:02.584998 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.584971    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c691124b-f854-447e-893f-5db54c13a05c-console-serving-cert\") pod \"console-6847b7689-v74xn\" (UID: \"c691124b-f854-447e-893f-5db54c13a05c\") " pod="openshift-console/console-6847b7689-v74xn"
Sep 15 16:28:02.590588 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.590556    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfpxg\" (UniqueName: \"kubernetes.io/projected/c691124b-f854-447e-893f-5db54c13a05c-kube-api-access-nfpxg\") pod \"console-6847b7689-v74xn\" (UID: \"c691124b-f854-447e-893f-5db54c13a05c\") " pod="openshift-console/console-6847b7689-v74xn"
Sep 15 16:28:02.762524 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:02.762500    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6847b7689-v74xn"
Sep 15 16:28:03.065029 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:03.064961    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6847b7689-v74xn"]
Sep 15 16:28:03.067693 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:28:03.067664    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc691124b_f854_447e_893f_5db54c13a05c.slice/crio-8bd1a0e182c914498b45a552a2069421e154db219029cf81c13c2c95cae92d4c WatchSource:0}: Error finding container 8bd1a0e182c914498b45a552a2069421e154db219029cf81c13c2c95cae92d4c: Status 404 returned error can't find the container with id 8bd1a0e182c914498b45a552a2069421e154db219029cf81c13c2c95cae92d4c
Sep 15 16:28:03.353806 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:03.353724    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6847b7689-v74xn" event={"ID":"c691124b-f854-447e-893f-5db54c13a05c","Type":"ContainerStarted","Data":"8bd1a0e182c914498b45a552a2069421e154db219029cf81c13c2c95cae92d4c"}
Sep 15 16:28:05.217279 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:05.217252    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-sqdn2"
Sep 15 16:28:06.366084 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:06.366052    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6847b7689-v74xn" event={"ID":"c691124b-f854-447e-893f-5db54c13a05c","Type":"ContainerStarted","Data":"d92c0a62762736312253f1327f312b9eaecab6b34868b023cb3053e7b78d4ba4"}
Sep 15 16:28:06.397672 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:06.397626    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6847b7689-v74xn" podStartSLOduration=1.292345339 podStartE2EDuration="4.39761118s" totalImagesPullingTime="3.105265841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 16:28:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:28:06.396553945 +0000 UTC m=+103.026208297" watchObservedRunningTime="2026-09-15 16:28:06.39761118 +0000 UTC m=+103.027265530"
Sep 15 16:28:07.354763 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:07.354738    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-759d5984cf-cx47r"
Sep 15 16:28:12.763454 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:12.763421    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6847b7689-v74xn"
Sep 15 16:28:12.763864 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:12.763641    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6847b7689-v74xn"
Sep 15 16:28:12.767903 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:12.767883    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6847b7689-v74xn"
Sep 15 16:28:13.395394 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:13.395373    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6847b7689-v74xn"
Sep 15 16:28:15.625777 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:28:15.625744    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2kxsk_84da0f72-32d0-4f59-9cad-f2c59cc71a0b/serve-healthcheck-canary/0.log"
Sep 15 16:29:14.456551 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:14.456524    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 15 16:29:14.457066 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:14.456815    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerName="alertmanager" containerID="cri-o://cc69740c1f7dbc3d7b970647cf78a3b4baba256969cfa7560096298a87e18e56" gracePeriod=120
Sep 15 16:29:14.457066 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:14.456857    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerName="kube-rbac-proxy-metric" containerID="cri-o://a609ecbe95e9aba12953a63b964274cf60bf1978c9768c8aa483e30adcfd6adc" gracePeriod=120
Sep 15 16:29:14.457066 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:14.456881    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerName="kube-rbac-proxy-web" containerID="cri-o://903e3c8a6d908cf8120b7933d600e17a097193917ed56932e208622d33d511d9" gracePeriod=120
Sep 15 16:29:14.457066 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:14.456922    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerName="prom-label-proxy" containerID="cri-o://e6dd289119f786b25398b2122cf49df1bdd3bc10b6b40bd3262a9d9c7d480c2b" gracePeriod=120
Sep 15 16:29:14.457066 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:14.456980    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerName="kube-rbac-proxy" containerID="cri-o://259487950e76627062b673e90c62b2944ef97d5701a6322985f39c7dd5bf1541" gracePeriod=120
Sep 15 16:29:14.457066 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:14.456919    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerName="config-reloader" containerID="cri-o://c1de56608d4b0b4253d092c07ea108f05f67152444ffa9d7701f973185e91687" gracePeriod=120
Sep 15 16:29:14.600119 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:14.600096    2579 generic.go:356] "Generic (PLEG): container finished" podID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerID="e6dd289119f786b25398b2122cf49df1bdd3bc10b6b40bd3262a9d9c7d480c2b" exitCode=0
Sep 15 16:29:14.600119 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:14.600115    2579 generic.go:356] "Generic (PLEG): container finished" podID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerID="259487950e76627062b673e90c62b2944ef97d5701a6322985f39c7dd5bf1541" exitCode=0
Sep 15 16:29:14.600119 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:14.600122    2579 generic.go:356] "Generic (PLEG): container finished" podID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerID="c1de56608d4b0b4253d092c07ea108f05f67152444ffa9d7701f973185e91687" exitCode=0
Sep 15 16:29:14.600260 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:14.600128    2579 generic.go:356] "Generic (PLEG): container finished" podID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerID="cc69740c1f7dbc3d7b970647cf78a3b4baba256969cfa7560096298a87e18e56" exitCode=0
Sep 15 16:29:14.600260 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:14.600145    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"15efc309-7d8f-4b51-92e2-3c8e09ca50b0","Type":"ContainerDied","Data":"e6dd289119f786b25398b2122cf49df1bdd3bc10b6b40bd3262a9d9c7d480c2b"}
Sep 15 16:29:14.600260 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:14.600195    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"15efc309-7d8f-4b51-92e2-3c8e09ca50b0","Type":"ContainerDied","Data":"259487950e76627062b673e90c62b2944ef97d5701a6322985f39c7dd5bf1541"}
Sep 15 16:29:14.600260 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:14.600221    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"15efc309-7d8f-4b51-92e2-3c8e09ca50b0","Type":"ContainerDied","Data":"c1de56608d4b0b4253d092c07ea108f05f67152444ffa9d7701f973185e91687"}
Sep 15 16:29:14.600260 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:14.600242    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"15efc309-7d8f-4b51-92e2-3c8e09ca50b0","Type":"ContainerDied","Data":"cc69740c1f7dbc3d7b970647cf78a3b4baba256969cfa7560096298a87e18e56"}
Sep 15 16:29:15.607907 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.607876    2579 generic.go:356] "Generic (PLEG): container finished" podID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerID="a609ecbe95e9aba12953a63b964274cf60bf1978c9768c8aa483e30adcfd6adc" exitCode=0
Sep 15 16:29:15.607907 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.607902    2579 generic.go:356] "Generic (PLEG): container finished" podID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerID="903e3c8a6d908cf8120b7933d600e17a097193917ed56932e208622d33d511d9" exitCode=0
Sep 15 16:29:15.608324 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.607929    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"15efc309-7d8f-4b51-92e2-3c8e09ca50b0","Type":"ContainerDied","Data":"a609ecbe95e9aba12953a63b964274cf60bf1978c9768c8aa483e30adcfd6adc"}
Sep 15 16:29:15.608324 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.607995    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"15efc309-7d8f-4b51-92e2-3c8e09ca50b0","Type":"ContainerDied","Data":"903e3c8a6d908cf8120b7933d600e17a097193917ed56932e208622d33d511d9"}
Sep 15 16:29:15.694873 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.694847    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:15.870972 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.870900    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-alertmanager-trusted-ca-bundle\") pod \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") "
Sep 15 16:29:15.870972 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.870942    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-metrics-client-ca\") pod \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") "
Sep 15 16:29:15.870972 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.870970    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-tls-assets\") pod \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") "
Sep 15 16:29:15.871201 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.870999    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-config-out\") pod \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") "
Sep 15 16:29:15.871201 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.871030    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-web-config\") pod \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") "
Sep 15 16:29:15.871201 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.871071    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-config-volume\") pod \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") "
Sep 15 16:29:15.871201 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.871095    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-alertmanager-main-db\") pod \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") "
Sep 15 16:29:15.871201 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.871119    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-secret-alertmanager-kube-rbac-proxy\") pod \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") "
Sep 15 16:29:15.871201 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.871149    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rj6lj\" (UniqueName: \"kubernetes.io/projected/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-kube-api-access-rj6lj\") pod \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") "
Sep 15 16:29:15.871201 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.871192    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-secret-alertmanager-main-tls\") pod \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") "
Sep 15 16:29:15.871532 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.871225    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-secret-alertmanager-kube-rbac-proxy-web\") pod \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") "
Sep 15 16:29:15.871532 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.871250    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-cluster-tls-config\") pod \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") "
Sep 15 16:29:15.871532 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.871286    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\" (UID: \"15efc309-7d8f-4b51-92e2-3c8e09ca50b0\") "
Sep 15 16:29:15.871532 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.871338    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "15efc309-7d8f-4b51-92e2-3c8e09ca50b0" (UID: "15efc309-7d8f-4b51-92e2-3c8e09ca50b0"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:29:15.871532 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.871348    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "15efc309-7d8f-4b51-92e2-3c8e09ca50b0" (UID: "15efc309-7d8f-4b51-92e2-3c8e09ca50b0"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:29:15.871532 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.871513    2579 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:29:15.871532 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.871533    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-metrics-client-ca\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:29:15.872478 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.872447    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "15efc309-7d8f-4b51-92e2-3c8e09ca50b0" (UID: "15efc309-7d8f-4b51-92e2-3c8e09ca50b0"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 16:29:15.874295 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.874273    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-config-out" (OuterVolumeSpecName: "config-out") pod "15efc309-7d8f-4b51-92e2-3c8e09ca50b0" (UID: "15efc309-7d8f-4b51-92e2-3c8e09ca50b0"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 16:29:15.874427 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.874311    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-config-volume" (OuterVolumeSpecName: "config-volume") pod "15efc309-7d8f-4b51-92e2-3c8e09ca50b0" (UID: "15efc309-7d8f-4b51-92e2-3c8e09ca50b0"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 16:29:15.874427 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.874410    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "15efc309-7d8f-4b51-92e2-3c8e09ca50b0" (UID: "15efc309-7d8f-4b51-92e2-3c8e09ca50b0"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 16:29:15.874879 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.874842    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "15efc309-7d8f-4b51-92e2-3c8e09ca50b0" (UID: "15efc309-7d8f-4b51-92e2-3c8e09ca50b0"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 16:29:15.875016 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.874990    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-kube-api-access-rj6lj" (OuterVolumeSpecName: "kube-api-access-rj6lj") pod "15efc309-7d8f-4b51-92e2-3c8e09ca50b0" (UID: "15efc309-7d8f-4b51-92e2-3c8e09ca50b0"). InnerVolumeSpecName "kube-api-access-rj6lj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 16:29:15.875106 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.874989    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "15efc309-7d8f-4b51-92e2-3c8e09ca50b0" (UID: "15efc309-7d8f-4b51-92e2-3c8e09ca50b0"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 16:29:15.875106 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.875034    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "15efc309-7d8f-4b51-92e2-3c8e09ca50b0" (UID: "15efc309-7d8f-4b51-92e2-3c8e09ca50b0"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 16:29:15.875815 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.875764    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "15efc309-7d8f-4b51-92e2-3c8e09ca50b0" (UID: "15efc309-7d8f-4b51-92e2-3c8e09ca50b0"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 16:29:15.879689 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.879661    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "15efc309-7d8f-4b51-92e2-3c8e09ca50b0" (UID: "15efc309-7d8f-4b51-92e2-3c8e09ca50b0"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 16:29:15.885156 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.885134    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-web-config" (OuterVolumeSpecName: "web-config") pod "15efc309-7d8f-4b51-92e2-3c8e09ca50b0" (UID: "15efc309-7d8f-4b51-92e2-3c8e09ca50b0"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 16:29:15.972569 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.972549    2579 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-config-volume\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:29:15.972642 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.972570    2579 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-alertmanager-main-db\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:29:15.972642 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.972579    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:29:15.972642 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.972587    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rj6lj\" (UniqueName: \"kubernetes.io/projected/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-kube-api-access-rj6lj\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:29:15.972642 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.972596    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-secret-alertmanager-main-tls\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:29:15.972642 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.972604    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:29:15.972642 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.972612    2579 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-cluster-tls-config\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:29:15.972642 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.972621    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:29:15.972642 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.972630    2579 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-tls-assets\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:29:15.972642 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.972637    2579 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-config-out\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:29:15.972642 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:15.972644    2579 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/15efc309-7d8f-4b51-92e2-3c8e09ca50b0-web-config\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:29:16.615618 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.615588    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"15efc309-7d8f-4b51-92e2-3c8e09ca50b0","Type":"ContainerDied","Data":"203e0aa6a06bd551b392c0fe2277f9a4b7c59a4611bb7dab99678e3401c90eb7"}
Sep 15 16:29:16.615907 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.615636    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.615907 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.615705    2579 scope.go:117] "RemoveContainer" containerID="e6dd289119f786b25398b2122cf49df1bdd3bc10b6b40bd3262a9d9c7d480c2b"
Sep 15 16:29:16.623170 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.623153    2579 scope.go:117] "RemoveContainer" containerID="a609ecbe95e9aba12953a63b964274cf60bf1978c9768c8aa483e30adcfd6adc"
Sep 15 16:29:16.629978 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.629962    2579 scope.go:117] "RemoveContainer" containerID="259487950e76627062b673e90c62b2944ef97d5701a6322985f39c7dd5bf1541"
Sep 15 16:29:16.636209 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.636189    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 15 16:29:16.636893 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.636869    2579 scope.go:117] "RemoveContainer" containerID="903e3c8a6d908cf8120b7933d600e17a097193917ed56932e208622d33d511d9"
Sep 15 16:29:16.640146 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.640125    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 15 16:29:16.643827 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.643775    2579 scope.go:117] "RemoveContainer" containerID="c1de56608d4b0b4253d092c07ea108f05f67152444ffa9d7701f973185e91687"
Sep 15 16:29:16.650027 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.650013    2579 scope.go:117] "RemoveContainer" containerID="cc69740c1f7dbc3d7b970647cf78a3b4baba256969cfa7560096298a87e18e56"
Sep 15 16:29:16.656360 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.656343    2579 scope.go:117] "RemoveContainer" containerID="6a1a9295b654cb0764dbfa7ba10f810695adef741acada36b6aec515a290f381"
Sep 15 16:29:16.664774 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.664755    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 15 16:29:16.665031 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.665018    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerName="config-reloader"
Sep 15 16:29:16.665077 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.665033    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerName="config-reloader"
Sep 15 16:29:16.665077 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.665043    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerName="prom-label-proxy"
Sep 15 16:29:16.665077 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.665048    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerName="prom-label-proxy"
Sep 15 16:29:16.665077 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.665057    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerName="init-config-reloader"
Sep 15 16:29:16.665077 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.665062    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerName="init-config-reloader"
Sep 15 16:29:16.665077 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.665071    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerName="kube-rbac-proxy"
Sep 15 16:29:16.665077 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.665077    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerName="kube-rbac-proxy"
Sep 15 16:29:16.665268 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.665084    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerName="alertmanager"
Sep 15 16:29:16.665268 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.665089    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerName="alertmanager"
Sep 15 16:29:16.665268 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.665096    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerName="kube-rbac-proxy-web"
Sep 15 16:29:16.665268 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.665101    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerName="kube-rbac-proxy-web"
Sep 15 16:29:16.665268 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.665110    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerName="kube-rbac-proxy-metric"
Sep 15 16:29:16.665268 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.665115    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerName="kube-rbac-proxy-metric"
Sep 15 16:29:16.665268 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.665157    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerName="alertmanager"
Sep 15 16:29:16.665268 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.665167    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerName="config-reloader"
Sep 15 16:29:16.665268 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.665172    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerName="kube-rbac-proxy-web"
Sep 15 16:29:16.665268 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.665177    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerName="kube-rbac-proxy-metric"
Sep 15 16:29:16.665268 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.665183    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerName="kube-rbac-proxy"
Sep 15 16:29:16.665268 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.665187    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" containerName="prom-label-proxy"
Sep 15 16:29:16.669605 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.669591    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.671991 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.671970    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 15 16:29:16.672071 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.671991    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-pbbkh\""
Sep 15 16:29:16.672071 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.672006    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 15 16:29:16.672071 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.672050    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 15 16:29:16.672566 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.672546    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 15 16:29:16.672685 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.672554    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 15 16:29:16.672685 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.672612    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 15 16:29:16.672838 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.672682    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 15 16:29:16.672937 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.672924    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 15 16:29:16.677642 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.677624    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 15 16:29:16.685690 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.685671    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 15 16:29:16.777853 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.777826    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bb3350d5-70d9-4dcd-a871-b74b9058f15a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.777949 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.777858    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb3350d5-70d9-4dcd-a871-b74b9058f15a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.777949 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.777894    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bb3350d5-70d9-4dcd-a871-b74b9058f15a-web-config\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.777949 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.777917    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bb3350d5-70d9-4dcd-a871-b74b9058f15a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.777949 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.777944    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bb3350d5-70d9-4dcd-a871-b74b9058f15a-config-out\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.778156 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.777969    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bb3350d5-70d9-4dcd-a871-b74b9058f15a-config-volume\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.778156 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.777991    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl85n\" (UniqueName: \"kubernetes.io/projected/bb3350d5-70d9-4dcd-a871-b74b9058f15a-kube-api-access-kl85n\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.778156 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.778013    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bb3350d5-70d9-4dcd-a871-b74b9058f15a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.778156 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.778042    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bb3350d5-70d9-4dcd-a871-b74b9058f15a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.778156 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.778111    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bb3350d5-70d9-4dcd-a871-b74b9058f15a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.778156 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.778144    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bb3350d5-70d9-4dcd-a871-b74b9058f15a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.778361 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.778166    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bb3350d5-70d9-4dcd-a871-b74b9058f15a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.778361 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.778190    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb3350d5-70d9-4dcd-a871-b74b9058f15a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.879228 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.879175    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bb3350d5-70d9-4dcd-a871-b74b9058f15a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.879228 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.879210    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bb3350d5-70d9-4dcd-a871-b74b9058f15a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.879362 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.879235    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bb3350d5-70d9-4dcd-a871-b74b9058f15a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.879452 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.879430    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb3350d5-70d9-4dcd-a871-b74b9058f15a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.879514 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.879474    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bb3350d5-70d9-4dcd-a871-b74b9058f15a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.879514 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.879500    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb3350d5-70d9-4dcd-a871-b74b9058f15a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.879611 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.879539    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bb3350d5-70d9-4dcd-a871-b74b9058f15a-web-config\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.879611 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.879560    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bb3350d5-70d9-4dcd-a871-b74b9058f15a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.879611 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.879607    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bb3350d5-70d9-4dcd-a871-b74b9058f15a-config-out\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.879751 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.879634    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bb3350d5-70d9-4dcd-a871-b74b9058f15a-config-volume\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.879751 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.879656    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl85n\" (UniqueName: \"kubernetes.io/projected/bb3350d5-70d9-4dcd-a871-b74b9058f15a-kube-api-access-kl85n\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.879751 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.879676    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bb3350d5-70d9-4dcd-a871-b74b9058f15a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.879751 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.879703    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bb3350d5-70d9-4dcd-a871-b74b9058f15a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.880476 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.880232    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bb3350d5-70d9-4dcd-a871-b74b9058f15a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.880476 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.880255    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb3350d5-70d9-4dcd-a871-b74b9058f15a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.880476 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.880374    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb3350d5-70d9-4dcd-a871-b74b9058f15a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.882610 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.882309    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bb3350d5-70d9-4dcd-a871-b74b9058f15a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.882610 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.882556    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bb3350d5-70d9-4dcd-a871-b74b9058f15a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.882610 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.882562    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bb3350d5-70d9-4dcd-a871-b74b9058f15a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.882810 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.882774    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bb3350d5-70d9-4dcd-a871-b74b9058f15a-config-volume\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.882870 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.882810    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bb3350d5-70d9-4dcd-a871-b74b9058f15a-web-config\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.883137 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.883112    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bb3350d5-70d9-4dcd-a871-b74b9058f15a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.883316 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.883295    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bb3350d5-70d9-4dcd-a871-b74b9058f15a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.883375 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.883296    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bb3350d5-70d9-4dcd-a871-b74b9058f15a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.884329 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.884316    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bb3350d5-70d9-4dcd-a871-b74b9058f15a-config-out\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.888113 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.888093    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl85n\" (UniqueName: \"kubernetes.io/projected/bb3350d5-70d9-4dcd-a871-b74b9058f15a-kube-api-access-kl85n\") pod \"alertmanager-main-0\" (UID: \"bb3350d5-70d9-4dcd-a871-b74b9058f15a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:16.978895 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:16.978867    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 15 16:29:17.255681 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:17.255657    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 15 16:29:17.259318 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:29:17.259295    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb3350d5_70d9_4dcd_a871_b74b9058f15a.slice/crio-934a2b658ff36c3d50575fb85cc71a2691c65a703acb6c71e4e65bdf87a0a733 WatchSource:0}: Error finding container 934a2b658ff36c3d50575fb85cc71a2691c65a703acb6c71e4e65bdf87a0a733: Status 404 returned error can't find the container with id 934a2b658ff36c3d50575fb85cc71a2691c65a703acb6c71e4e65bdf87a0a733
Sep 15 16:29:17.620216 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:17.620152    2579 generic.go:356] "Generic (PLEG): container finished" podID="bb3350d5-70d9-4dcd-a871-b74b9058f15a" containerID="9c3c6c0fe57adb3afe2bbe597c1c30a536effa314fab31afbf11785f36707c65" exitCode=0
Sep 15 16:29:17.620515 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:17.620247    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb3350d5-70d9-4dcd-a871-b74b9058f15a","Type":"ContainerDied","Data":"9c3c6c0fe57adb3afe2bbe597c1c30a536effa314fab31afbf11785f36707c65"}
Sep 15 16:29:17.620515 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:17.620351    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb3350d5-70d9-4dcd-a871-b74b9058f15a","Type":"ContainerStarted","Data":"934a2b658ff36c3d50575fb85cc71a2691c65a703acb6c71e4e65bdf87a0a733"}
Sep 15 16:29:17.907912 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:17.907890    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15efc309-7d8f-4b51-92e2-3c8e09ca50b0" path="/var/lib/kubelet/pods/15efc309-7d8f-4b51-92e2-3c8e09ca50b0/volumes"
Sep 15 16:29:18.484771 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.484746    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7b848cd67c-kqz4k"]
Sep 15 16:29:18.487999 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.487983    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k"
Sep 15 16:29:18.490480 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.490456    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 15 16:29:18.490594 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.490483    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 15 16:29:18.490594 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.490501    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 15 16:29:18.490594 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.490505    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-wc4cr\""
Sep 15 16:29:18.490594 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.490490    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 15 16:29:18.490890 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.490846    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 15 16:29:18.495353 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.495338    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 15 16:29:18.501454 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.501437    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7b848cd67c-kqz4k"]
Sep 15 16:29:18.591518 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.591495    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a8e15716-7df1-488d-8adc-44a27496147f-telemeter-client-tls\") pod \"telemeter-client-7b848cd67c-kqz4k\" (UID: \"a8e15716-7df1-488d-8adc-44a27496147f\") " pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k"
Sep 15 16:29:18.591593 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.591528    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8e15716-7df1-488d-8adc-44a27496147f-serving-certs-ca-bundle\") pod \"telemeter-client-7b848cd67c-kqz4k\" (UID: \"a8e15716-7df1-488d-8adc-44a27496147f\") " pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k"
Sep 15 16:29:18.591593 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.591547    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8e15716-7df1-488d-8adc-44a27496147f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b848cd67c-kqz4k\" (UID: \"a8e15716-7df1-488d-8adc-44a27496147f\") " pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k"
Sep 15 16:29:18.591593 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.591572    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a8e15716-7df1-488d-8adc-44a27496147f-secret-telemeter-client\") pod \"telemeter-client-7b848cd67c-kqz4k\" (UID: \"a8e15716-7df1-488d-8adc-44a27496147f\") " pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k"
Sep 15 16:29:18.591758 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.591670    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a8e15716-7df1-488d-8adc-44a27496147f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b848cd67c-kqz4k\" (UID: \"a8e15716-7df1-488d-8adc-44a27496147f\") " pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k"
Sep 15 16:29:18.591758 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.591703    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhdxz\" (UniqueName: \"kubernetes.io/projected/a8e15716-7df1-488d-8adc-44a27496147f-kube-api-access-nhdxz\") pod \"telemeter-client-7b848cd67c-kqz4k\" (UID: \"a8e15716-7df1-488d-8adc-44a27496147f\") " pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k"
Sep 15 16:29:18.591758 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.591740    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a8e15716-7df1-488d-8adc-44a27496147f-federate-client-tls\") pod \"telemeter-client-7b848cd67c-kqz4k\" (UID: \"a8e15716-7df1-488d-8adc-44a27496147f\") " pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k"
Sep 15 16:29:18.591918 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.591806    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8e15716-7df1-488d-8adc-44a27496147f-metrics-client-ca\") pod \"telemeter-client-7b848cd67c-kqz4k\" (UID: \"a8e15716-7df1-488d-8adc-44a27496147f\") " pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k"
Sep 15 16:29:18.627850 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.627826    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb3350d5-70d9-4dcd-a871-b74b9058f15a","Type":"ContainerStarted","Data":"028871409aa9c9c64a2cfa83bba5d5740f130f270052bb3608849127c4612b6d"}
Sep 15 16:29:18.628109 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.627876    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb3350d5-70d9-4dcd-a871-b74b9058f15a","Type":"ContainerStarted","Data":"4444d2f799d457439437fa2f2e7c67ecf7f2683813c2ba8c511deae86701bfba"}
Sep 15 16:29:18.628109 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.627907    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb3350d5-70d9-4dcd-a871-b74b9058f15a","Type":"ContainerStarted","Data":"0035a97d2af132f7aadad6b62a55f8d9cec81ae8de9c9e24ed20dcbf43ef8362"}
Sep 15 16:29:18.628109 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.627933    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb3350d5-70d9-4dcd-a871-b74b9058f15a","Type":"ContainerStarted","Data":"f3eee94e463d3a2d44e2a5468c2600a16b1ead83b33321439a323df62651347a"}
Sep 15 16:29:18.628109 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.627957    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb3350d5-70d9-4dcd-a871-b74b9058f15a","Type":"ContainerStarted","Data":"c4a76d04cc946f4bc8d0d13936b01c872f2ea667ec9947a2fe15933164ad4eea"}
Sep 15 16:29:18.628109 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.627996    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb3350d5-70d9-4dcd-a871-b74b9058f15a","Type":"ContainerStarted","Data":"55f15b69b6fe0b70cafcd1860df8f111eb44a6826c728679fb9d7e12ea719476"}
Sep 15 16:29:18.652346 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.652306    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.637868794 podStartE2EDuration="2.652293543s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.424749ms" isStatefulPod=false podCreationTimestamp="2026-09-15 16:29:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:29:18.650881793 +0000 UTC m=+175.280536148" watchObservedRunningTime="2026-09-15 16:29:18.652293543 +0000 UTC m=+175.281947894"
Sep 15 16:29:18.692884 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.692864    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a8e15716-7df1-488d-8adc-44a27496147f-telemeter-client-tls\") pod \"telemeter-client-7b848cd67c-kqz4k\" (UID: \"a8e15716-7df1-488d-8adc-44a27496147f\") " pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k"
Sep 15 16:29:18.692985 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.692900    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8e15716-7df1-488d-8adc-44a27496147f-serving-certs-ca-bundle\") pod \"telemeter-client-7b848cd67c-kqz4k\" (UID: \"a8e15716-7df1-488d-8adc-44a27496147f\") " pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k"
Sep 15 16:29:18.692985 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.692918    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8e15716-7df1-488d-8adc-44a27496147f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b848cd67c-kqz4k\" (UID: \"a8e15716-7df1-488d-8adc-44a27496147f\") " pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k"
Sep 15 16:29:18.692985 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.692934    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a8e15716-7df1-488d-8adc-44a27496147f-secret-telemeter-client\") pod \"telemeter-client-7b848cd67c-kqz4k\" (UID: \"a8e15716-7df1-488d-8adc-44a27496147f\") " pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k"
Sep 15 16:29:18.692985 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.692963    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a8e15716-7df1-488d-8adc-44a27496147f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b848cd67c-kqz4k\" (UID: \"a8e15716-7df1-488d-8adc-44a27496147f\") " pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k"
Sep 15 16:29:18.693161 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.692986    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhdxz\" (UniqueName: \"kubernetes.io/projected/a8e15716-7df1-488d-8adc-44a27496147f-kube-api-access-nhdxz\") pod \"telemeter-client-7b848cd67c-kqz4k\" (UID: \"a8e15716-7df1-488d-8adc-44a27496147f\") " pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k"
Sep 15 16:29:18.693161 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.693017    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a8e15716-7df1-488d-8adc-44a27496147f-federate-client-tls\") pod \"telemeter-client-7b848cd67c-kqz4k\" (UID: \"a8e15716-7df1-488d-8adc-44a27496147f\") " pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k"
Sep 15 16:29:18.694275 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.693230    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8e15716-7df1-488d-8adc-44a27496147f-metrics-client-ca\") pod \"telemeter-client-7b848cd67c-kqz4k\" (UID: \"a8e15716-7df1-488d-8adc-44a27496147f\") " pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k"
Sep 15 16:29:18.694275 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.693829    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8e15716-7df1-488d-8adc-44a27496147f-serving-certs-ca-bundle\") pod \"telemeter-client-7b848cd67c-kqz4k\" (UID: \"a8e15716-7df1-488d-8adc-44a27496147f\") " pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k"
Sep 15 16:29:18.694275 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.694096    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8e15716-7df1-488d-8adc-44a27496147f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b848cd67c-kqz4k\" (UID: \"a8e15716-7df1-488d-8adc-44a27496147f\") " pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k"
Sep 15 16:29:18.694275 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.694132    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8e15716-7df1-488d-8adc-44a27496147f-metrics-client-ca\") pod \"telemeter-client-7b848cd67c-kqz4k\" (UID: \"a8e15716-7df1-488d-8adc-44a27496147f\") " pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k"
Sep 15 16:29:18.695738 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.695710    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a8e15716-7df1-488d-8adc-44a27496147f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b848cd67c-kqz4k\" (UID: \"a8e15716-7df1-488d-8adc-44a27496147f\") " pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k"
Sep 15 16:29:18.695841 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.695808    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a8e15716-7df1-488d-8adc-44a27496147f-secret-telemeter-client\") pod \"telemeter-client-7b848cd67c-kqz4k\" (UID: \"a8e15716-7df1-488d-8adc-44a27496147f\") " pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k"
Sep 15 16:29:18.695954 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.695931    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a8e15716-7df1-488d-8adc-44a27496147f-federate-client-tls\") pod \"telemeter-client-7b848cd67c-kqz4k\" (UID: \"a8e15716-7df1-488d-8adc-44a27496147f\") " pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k"
Sep 15 16:29:18.696239 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.696223    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a8e15716-7df1-488d-8adc-44a27496147f-telemeter-client-tls\") pod \"telemeter-client-7b848cd67c-kqz4k\" (UID: \"a8e15716-7df1-488d-8adc-44a27496147f\") " pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k"
Sep 15 16:29:18.700538 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.700521    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhdxz\" (UniqueName: \"kubernetes.io/projected/a8e15716-7df1-488d-8adc-44a27496147f-kube-api-access-nhdxz\") pod \"telemeter-client-7b848cd67c-kqz4k\" (UID: \"a8e15716-7df1-488d-8adc-44a27496147f\") " pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k"
Sep 15 16:29:18.797729 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:18.797513    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k"
Sep 15 16:29:19.083650 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:19.083584    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7b848cd67c-kqz4k"]
Sep 15 16:29:19.087000 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:29:19.086962    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8e15716_7df1_488d_8adc_44a27496147f.slice/crio-db2415efa0811ac301684ba128aa1c458ed68f9e6d86b3d358fce58f7ac4ddb1 WatchSource:0}: Error finding container db2415efa0811ac301684ba128aa1c458ed68f9e6d86b3d358fce58f7ac4ddb1: Status 404 returned error can't find the container with id db2415efa0811ac301684ba128aa1c458ed68f9e6d86b3d358fce58f7ac4ddb1
Sep 15 16:29:19.632186 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:19.632160    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k" event={"ID":"a8e15716-7df1-488d-8adc-44a27496147f","Type":"ContainerStarted","Data":"db2415efa0811ac301684ba128aa1c458ed68f9e6d86b3d358fce58f7ac4ddb1"}
Sep 15 16:29:21.642744 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:21.642723    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k" event={"ID":"a8e15716-7df1-488d-8adc-44a27496147f","Type":"ContainerStarted","Data":"7128a95d1347e273c5bacf4d9fa6a0a51634c4fdb03c16eee3a8e25901d358a2"}
Sep 15 16:29:21.643045 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:21.642817    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k" event={"ID":"a8e15716-7df1-488d-8adc-44a27496147f","Type":"ContainerStarted","Data":"2e7d9276b8863e956dc67d8631ebbe1dc28b015d4d51f275e62696e164e4cae0"}
Sep 15 16:29:22.648110 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:22.648074    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k" event={"ID":"a8e15716-7df1-488d-8adc-44a27496147f","Type":"ContainerStarted","Data":"18dfcf7304b7d4c1a5a9b2c66061787d1e35d81cffd5024d525efcdfa29e695b"}
Sep 15 16:29:22.672197 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:22.672155    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7b848cd67c-kqz4k" podStartSLOduration=2.270223246 podStartE2EDuration="4.672142269s" totalImagesPullingTime="2.401919023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 16:29:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:29:22.670523108 +0000 UTC m=+179.300177460" watchObservedRunningTime="2026-09-15 16:29:22.672142269 +0000 UTC m=+179.301796640"
Sep 15 16:29:22.760369 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:22.760345    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7fb8b847f-m8g5t"]
Sep 15 16:29:22.763396 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:22.763368    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fb8b847f-m8g5t"
Sep 15 16:29:22.775218 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:22.775198    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7fb8b847f-m8g5t"]
Sep 15 16:29:22.924418 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:22.924369    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d043aadb-a85e-4e22-b8fc-903d7ad472db-oauth-serving-cert\") pod \"console-7fb8b847f-m8g5t\" (UID: \"d043aadb-a85e-4e22-b8fc-903d7ad472db\") " pod="openshift-console/console-7fb8b847f-m8g5t"
Sep 15 16:29:22.924418 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:22.924401    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d043aadb-a85e-4e22-b8fc-903d7ad472db-console-serving-cert\") pod \"console-7fb8b847f-m8g5t\" (UID: \"d043aadb-a85e-4e22-b8fc-903d7ad472db\") " pod="openshift-console/console-7fb8b847f-m8g5t"
Sep 15 16:29:22.924536 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:22.924429    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d043aadb-a85e-4e22-b8fc-903d7ad472db-console-oauth-config\") pod \"console-7fb8b847f-m8g5t\" (UID: \"d043aadb-a85e-4e22-b8fc-903d7ad472db\") " pod="openshift-console/console-7fb8b847f-m8g5t"
Sep 15 16:29:22.924536 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:22.924444    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv7mg\" (UniqueName: \"kubernetes.io/projected/d043aadb-a85e-4e22-b8fc-903d7ad472db-kube-api-access-qv7mg\") pod \"console-7fb8b847f-m8g5t\" (UID: \"d043aadb-a85e-4e22-b8fc-903d7ad472db\") " pod="openshift-console/console-7fb8b847f-m8g5t"
Sep 15 16:29:22.924536 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:22.924493    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d043aadb-a85e-4e22-b8fc-903d7ad472db-console-config\") pod \"console-7fb8b847f-m8g5t\" (UID: \"d043aadb-a85e-4e22-b8fc-903d7ad472db\") " pod="openshift-console/console-7fb8b847f-m8g5t"
Sep 15 16:29:22.924536 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:22.924526    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d043aadb-a85e-4e22-b8fc-903d7ad472db-trusted-ca-bundle\") pod \"console-7fb8b847f-m8g5t\" (UID: \"d043aadb-a85e-4e22-b8fc-903d7ad472db\") " pod="openshift-console/console-7fb8b847f-m8g5t"
Sep 15 16:29:22.924659 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:22.924548    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d043aadb-a85e-4e22-b8fc-903d7ad472db-service-ca\") pod \"console-7fb8b847f-m8g5t\" (UID: \"d043aadb-a85e-4e22-b8fc-903d7ad472db\") " pod="openshift-console/console-7fb8b847f-m8g5t"
Sep 15 16:29:23.025759 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:23.025736    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d043aadb-a85e-4e22-b8fc-903d7ad472db-oauth-serving-cert\") pod \"console-7fb8b847f-m8g5t\" (UID: \"d043aadb-a85e-4e22-b8fc-903d7ad472db\") " pod="openshift-console/console-7fb8b847f-m8g5t"
Sep 15 16:29:23.025868 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:23.025770    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d043aadb-a85e-4e22-b8fc-903d7ad472db-console-serving-cert\") pod \"console-7fb8b847f-m8g5t\" (UID: \"d043aadb-a85e-4e22-b8fc-903d7ad472db\") " pod="openshift-console/console-7fb8b847f-m8g5t"
Sep 15 16:29:23.026100 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:23.026074    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d043aadb-a85e-4e22-b8fc-903d7ad472db-console-oauth-config\") pod \"console-7fb8b847f-m8g5t\" (UID: \"d043aadb-a85e-4e22-b8fc-903d7ad472db\") " pod="openshift-console/console-7fb8b847f-m8g5t"
Sep 15 16:29:23.026209 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:23.026138    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv7mg\" (UniqueName: \"kubernetes.io/projected/d043aadb-a85e-4e22-b8fc-903d7ad472db-kube-api-access-qv7mg\") pod \"console-7fb8b847f-m8g5t\" (UID: \"d043aadb-a85e-4e22-b8fc-903d7ad472db\") " pod="openshift-console/console-7fb8b847f-m8g5t"
Sep 15 16:29:23.026209 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:23.026176    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d043aadb-a85e-4e22-b8fc-903d7ad472db-console-config\") pod \"console-7fb8b847f-m8g5t\" (UID: \"d043aadb-a85e-4e22-b8fc-903d7ad472db\") " pod="openshift-console/console-7fb8b847f-m8g5t"
Sep 15 16:29:23.026209 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:23.026204    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d043aadb-a85e-4e22-b8fc-903d7ad472db-trusted-ca-bundle\") pod \"console-7fb8b847f-m8g5t\" (UID: \"d043aadb-a85e-4e22-b8fc-903d7ad472db\") " pod="openshift-console/console-7fb8b847f-m8g5t"
Sep 15 16:29:23.026360 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:23.026228    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d043aadb-a85e-4e22-b8fc-903d7ad472db-service-ca\") pod \"console-7fb8b847f-m8g5t\" (UID: \"d043aadb-a85e-4e22-b8fc-903d7ad472db\") " pod="openshift-console/console-7fb8b847f-m8g5t"
Sep 15 16:29:23.026437 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:23.026413    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d043aadb-a85e-4e22-b8fc-903d7ad472db-oauth-serving-cert\") pod \"console-7fb8b847f-m8g5t\" (UID: \"d043aadb-a85e-4e22-b8fc-903d7ad472db\") " pod="openshift-console/console-7fb8b847f-m8g5t"
Sep 15 16:29:23.027701 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:23.027676    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d043aadb-a85e-4e22-b8fc-903d7ad472db-console-config\") pod \"console-7fb8b847f-m8g5t\" (UID: \"d043aadb-a85e-4e22-b8fc-903d7ad472db\") " pod="openshift-console/console-7fb8b847f-m8g5t"
Sep 15 16:29:23.027841 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:23.027714    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d043aadb-a85e-4e22-b8fc-903d7ad472db-trusted-ca-bundle\") pod \"console-7fb8b847f-m8g5t\" (UID: \"d043aadb-a85e-4e22-b8fc-903d7ad472db\") " pod="openshift-console/console-7fb8b847f-m8g5t"
Sep 15 16:29:23.027943 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:23.027737    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d043aadb-a85e-4e22-b8fc-903d7ad472db-service-ca\") pod \"console-7fb8b847f-m8g5t\" (UID: \"d043aadb-a85e-4e22-b8fc-903d7ad472db\") " pod="openshift-console/console-7fb8b847f-m8g5t"
Sep 15 16:29:23.032515 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:23.032493    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d043aadb-a85e-4e22-b8fc-903d7ad472db-console-oauth-config\") pod \"console-7fb8b847f-m8g5t\" (UID: \"d043aadb-a85e-4e22-b8fc-903d7ad472db\") " pod="openshift-console/console-7fb8b847f-m8g5t"
Sep 15 16:29:23.032729 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:23.032708    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d043aadb-a85e-4e22-b8fc-903d7ad472db-console-serving-cert\") pod \"console-7fb8b847f-m8g5t\" (UID: \"d043aadb-a85e-4e22-b8fc-903d7ad472db\") " pod="openshift-console/console-7fb8b847f-m8g5t"
Sep 15 16:29:23.034921 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:23.034899    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv7mg\" (UniqueName: \"kubernetes.io/projected/d043aadb-a85e-4e22-b8fc-903d7ad472db-kube-api-access-qv7mg\") pod \"console-7fb8b847f-m8g5t\" (UID: \"d043aadb-a85e-4e22-b8fc-903d7ad472db\") " pod="openshift-console/console-7fb8b847f-m8g5t"
Sep 15 16:29:23.071417 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:23.071394    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fb8b847f-m8g5t"
Sep 15 16:29:23.341354 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:23.341332    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7fb8b847f-m8g5t"]
Sep 15 16:29:23.344423 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:29:23.344394    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd043aadb_a85e_4e22_b8fc_903d7ad472db.slice/crio-07510da6e3104754f0f5c0681ecfa6bf6da312fdcdfc2bac60a85a280e40b110 WatchSource:0}: Error finding container 07510da6e3104754f0f5c0681ecfa6bf6da312fdcdfc2bac60a85a280e40b110: Status 404 returned error can't find the container with id 07510da6e3104754f0f5c0681ecfa6bf6da312fdcdfc2bac60a85a280e40b110
Sep 15 16:29:23.652983 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:23.652921    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fb8b847f-m8g5t" event={"ID":"d043aadb-a85e-4e22-b8fc-903d7ad472db","Type":"ContainerStarted","Data":"2ce37e240d23bba2476412e8000f45374c1a775499460c0eeaadc2113995046b"}
Sep 15 16:29:23.652983 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:23.652966    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fb8b847f-m8g5t" event={"ID":"d043aadb-a85e-4e22-b8fc-903d7ad472db","Type":"ContainerStarted","Data":"07510da6e3104754f0f5c0681ecfa6bf6da312fdcdfc2bac60a85a280e40b110"}
Sep 15 16:29:23.670116 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:23.670074    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7fb8b847f-m8g5t" podStartSLOduration=1.670052599 podStartE2EDuration="1.670052599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 16:29:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:29:23.669862985 +0000 UTC m=+180.299517336" watchObservedRunningTime="2026-09-15 16:29:23.670052599 +0000 UTC m=+180.299706950"
Sep 15 16:29:33.071753 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:33.071726    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7fb8b847f-m8g5t"
Sep 15 16:29:33.072132 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:33.071777    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7fb8b847f-m8g5t"
Sep 15 16:29:33.075975 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:33.075957    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7fb8b847f-m8g5t"
Sep 15 16:29:33.690374 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:33.690353    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7fb8b847f-m8g5t"
Sep 15 16:29:33.736859 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:33.736838    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6847b7689-v74xn"]
Sep 15 16:29:58.759135 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:58.759068    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6847b7689-v74xn" podUID="c691124b-f854-447e-893f-5db54c13a05c" containerName="console" containerID="cri-o://d92c0a62762736312253f1327f312b9eaecab6b34868b023cb3053e7b78d4ba4" gracePeriod=15
Sep 15 16:29:59.005155 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.005132    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6847b7689-v74xn"
Sep 15 16:29:59.073524 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.073461    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c691124b-f854-447e-893f-5db54c13a05c-service-ca\") pod \"c691124b-f854-447e-893f-5db54c13a05c\" (UID: \"c691124b-f854-447e-893f-5db54c13a05c\") "
Sep 15 16:29:59.073524 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.073494    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c691124b-f854-447e-893f-5db54c13a05c-console-config\") pod \"c691124b-f854-447e-893f-5db54c13a05c\" (UID: \"c691124b-f854-447e-893f-5db54c13a05c\") "
Sep 15 16:29:59.073695 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.073529    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c691124b-f854-447e-893f-5db54c13a05c-oauth-serving-cert\") pod \"c691124b-f854-447e-893f-5db54c13a05c\" (UID: \"c691124b-f854-447e-893f-5db54c13a05c\") "
Sep 15 16:29:59.073695 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.073558    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfpxg\" (UniqueName: \"kubernetes.io/projected/c691124b-f854-447e-893f-5db54c13a05c-kube-api-access-nfpxg\") pod \"c691124b-f854-447e-893f-5db54c13a05c\" (UID: \"c691124b-f854-447e-893f-5db54c13a05c\") "
Sep 15 16:29:59.073695 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.073589    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c691124b-f854-447e-893f-5db54c13a05c-console-oauth-config\") pod \"c691124b-f854-447e-893f-5db54c13a05c\" (UID: \"c691124b-f854-447e-893f-5db54c13a05c\") "
Sep 15 16:29:59.073695 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.073634    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c691124b-f854-447e-893f-5db54c13a05c-console-serving-cert\") pod \"c691124b-f854-447e-893f-5db54c13a05c\" (UID: \"c691124b-f854-447e-893f-5db54c13a05c\") "
Sep 15 16:29:59.073695 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.073683    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c691124b-f854-447e-893f-5db54c13a05c-trusted-ca-bundle\") pod \"c691124b-f854-447e-893f-5db54c13a05c\" (UID: \"c691124b-f854-447e-893f-5db54c13a05c\") "
Sep 15 16:29:59.073952 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.073926    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c691124b-f854-447e-893f-5db54c13a05c-console-config" (OuterVolumeSpecName: "console-config") pod "c691124b-f854-447e-893f-5db54c13a05c" (UID: "c691124b-f854-447e-893f-5db54c13a05c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:29:59.074048 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.073934    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c691124b-f854-447e-893f-5db54c13a05c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c691124b-f854-447e-893f-5db54c13a05c" (UID: "c691124b-f854-447e-893f-5db54c13a05c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:29:59.074138 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.073938    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c691124b-f854-447e-893f-5db54c13a05c-service-ca" (OuterVolumeSpecName: "service-ca") pod "c691124b-f854-447e-893f-5db54c13a05c" (UID: "c691124b-f854-447e-893f-5db54c13a05c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:29:59.074239 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.074184    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c691124b-f854-447e-893f-5db54c13a05c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c691124b-f854-447e-893f-5db54c13a05c" (UID: "c691124b-f854-447e-893f-5db54c13a05c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:29:59.075920 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.075899    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c691124b-f854-447e-893f-5db54c13a05c-kube-api-access-nfpxg" (OuterVolumeSpecName: "kube-api-access-nfpxg") pod "c691124b-f854-447e-893f-5db54c13a05c" (UID: "c691124b-f854-447e-893f-5db54c13a05c"). InnerVolumeSpecName "kube-api-access-nfpxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 16:29:59.076013 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.075956    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c691124b-f854-447e-893f-5db54c13a05c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c691124b-f854-447e-893f-5db54c13a05c" (UID: "c691124b-f854-447e-893f-5db54c13a05c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 16:29:59.076069 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.076034    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c691124b-f854-447e-893f-5db54c13a05c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c691124b-f854-447e-893f-5db54c13a05c" (UID: "c691124b-f854-447e-893f-5db54c13a05c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 16:29:59.174616 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.174596    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c691124b-f854-447e-893f-5db54c13a05c-console-config\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:29:59.174616 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.174614    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c691124b-f854-447e-893f-5db54c13a05c-oauth-serving-cert\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:29:59.174736 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.174624    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfpxg\" (UniqueName: \"kubernetes.io/projected/c691124b-f854-447e-893f-5db54c13a05c-kube-api-access-nfpxg\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:29:59.174736 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.174632    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c691124b-f854-447e-893f-5db54c13a05c-console-oauth-config\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:29:59.174736 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.174640    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c691124b-f854-447e-893f-5db54c13a05c-console-serving-cert\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:29:59.174736 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.174650    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c691124b-f854-447e-893f-5db54c13a05c-trusted-ca-bundle\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:29:59.174736 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.174657    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c691124b-f854-447e-893f-5db54c13a05c-service-ca\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:29:59.775912 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.775887    2579 generic.go:356] "Generic (PLEG): container finished" podID="c691124b-f854-447e-893f-5db54c13a05c" containerID="d92c0a62762736312253f1327f312b9eaecab6b34868b023cb3053e7b78d4ba4" exitCode=0
Sep 15 16:29:59.776236 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.775934    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6847b7689-v74xn"
Sep 15 16:29:59.776236 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.775943    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6847b7689-v74xn" event={"ID":"c691124b-f854-447e-893f-5db54c13a05c","Type":"ContainerDied","Data":"d92c0a62762736312253f1327f312b9eaecab6b34868b023cb3053e7b78d4ba4"}
Sep 15 16:29:59.776236 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.775981    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6847b7689-v74xn" event={"ID":"c691124b-f854-447e-893f-5db54c13a05c","Type":"ContainerDied","Data":"8bd1a0e182c914498b45a552a2069421e154db219029cf81c13c2c95cae92d4c"}
Sep 15 16:29:59.776236 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.775998    2579 scope.go:117] "RemoveContainer" containerID="d92c0a62762736312253f1327f312b9eaecab6b34868b023cb3053e7b78d4ba4"
Sep 15 16:29:59.784064 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.784050    2579 scope.go:117] "RemoveContainer" containerID="d92c0a62762736312253f1327f312b9eaecab6b34868b023cb3053e7b78d4ba4"
Sep 15 16:29:59.784300 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:29:59.784283    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d92c0a62762736312253f1327f312b9eaecab6b34868b023cb3053e7b78d4ba4\": container with ID starting with d92c0a62762736312253f1327f312b9eaecab6b34868b023cb3053e7b78d4ba4 not found: ID does not exist" containerID="d92c0a62762736312253f1327f312b9eaecab6b34868b023cb3053e7b78d4ba4"
Sep 15 16:29:59.784346 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.784308    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d92c0a62762736312253f1327f312b9eaecab6b34868b023cb3053e7b78d4ba4"} err="failed to get container status \"d92c0a62762736312253f1327f312b9eaecab6b34868b023cb3053e7b78d4ba4\": rpc error: code = NotFound desc = could not find container \"d92c0a62762736312253f1327f312b9eaecab6b34868b023cb3053e7b78d4ba4\": container with ID starting with d92c0a62762736312253f1327f312b9eaecab6b34868b023cb3053e7b78d4ba4 not found: ID does not exist"
Sep 15 16:29:59.795889 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.795867    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6847b7689-v74xn"]
Sep 15 16:29:59.799821 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.799802    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6847b7689-v74xn"]
Sep 15 16:29:59.907565 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:29:59.907544    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c691124b-f854-447e-893f-5db54c13a05c" path="/var/lib/kubelet/pods/c691124b-f854-447e-893f-5db54c13a05c/volumes"
Sep 15 16:30:31.449124 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:31.449093    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56946cb5cf-n2zjq"]
Sep 15 16:30:31.449529 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:31.449359    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c691124b-f854-447e-893f-5db54c13a05c" containerName="console"
Sep 15 16:30:31.449529 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:31.449368    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c691124b-f854-447e-893f-5db54c13a05c" containerName="console"
Sep 15 16:30:31.449529 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:31.449423    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c691124b-f854-447e-893f-5db54c13a05c" containerName="console"
Sep 15 16:30:31.453427 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:31.453409    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56946cb5cf-n2zjq"
Sep 15 16:30:31.465447 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:31.465392    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56946cb5cf-n2zjq"]
Sep 15 16:30:31.594224 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:31.594197    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/09914dc8-1995-4417-8832-26944d93fb49-console-serving-cert\") pod \"console-56946cb5cf-n2zjq\" (UID: \"09914dc8-1995-4417-8832-26944d93fb49\") " pod="openshift-console/console-56946cb5cf-n2zjq"
Sep 15 16:30:31.594336 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:31.594230    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/09914dc8-1995-4417-8832-26944d93fb49-service-ca\") pod \"console-56946cb5cf-n2zjq\" (UID: \"09914dc8-1995-4417-8832-26944d93fb49\") " pod="openshift-console/console-56946cb5cf-n2zjq"
Sep 15 16:30:31.594336 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:31.594248    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09914dc8-1995-4417-8832-26944d93fb49-trusted-ca-bundle\") pod \"console-56946cb5cf-n2zjq\" (UID: \"09914dc8-1995-4417-8832-26944d93fb49\") " pod="openshift-console/console-56946cb5cf-n2zjq"
Sep 15 16:30:31.594336 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:31.594265    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/09914dc8-1995-4417-8832-26944d93fb49-console-oauth-config\") pod \"console-56946cb5cf-n2zjq\" (UID: \"09914dc8-1995-4417-8832-26944d93fb49\") " pod="openshift-console/console-56946cb5cf-n2zjq"
Sep 15 16:30:31.594336 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:31.594299    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/09914dc8-1995-4417-8832-26944d93fb49-oauth-serving-cert\") pod \"console-56946cb5cf-n2zjq\" (UID: \"09914dc8-1995-4417-8832-26944d93fb49\") " pod="openshift-console/console-56946cb5cf-n2zjq"
Sep 15 16:30:31.594509 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:31.594352    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/09914dc8-1995-4417-8832-26944d93fb49-console-config\") pod \"console-56946cb5cf-n2zjq\" (UID: \"09914dc8-1995-4417-8832-26944d93fb49\") " pod="openshift-console/console-56946cb5cf-n2zjq"
Sep 15 16:30:31.594509 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:31.594417    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l2b2\" (UniqueName: \"kubernetes.io/projected/09914dc8-1995-4417-8832-26944d93fb49-kube-api-access-7l2b2\") pod \"console-56946cb5cf-n2zjq\" (UID: \"09914dc8-1995-4417-8832-26944d93fb49\") " pod="openshift-console/console-56946cb5cf-n2zjq"
Sep 15 16:30:31.695475 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:31.695455    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/09914dc8-1995-4417-8832-26944d93fb49-console-config\") pod \"console-56946cb5cf-n2zjq\" (UID: \"09914dc8-1995-4417-8832-26944d93fb49\") " pod="openshift-console/console-56946cb5cf-n2zjq"
Sep 15 16:30:31.695565 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:31.695484    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l2b2\" (UniqueName: \"kubernetes.io/projected/09914dc8-1995-4417-8832-26944d93fb49-kube-api-access-7l2b2\") pod \"console-56946cb5cf-n2zjq\" (UID: \"09914dc8-1995-4417-8832-26944d93fb49\") " pod="openshift-console/console-56946cb5cf-n2zjq"
Sep 15 16:30:31.695565 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:31.695512    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/09914dc8-1995-4417-8832-26944d93fb49-console-serving-cert\") pod \"console-56946cb5cf-n2zjq\" (UID: \"09914dc8-1995-4417-8832-26944d93fb49\") " pod="openshift-console/console-56946cb5cf-n2zjq"
Sep 15 16:30:31.695565 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:31.695532    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/09914dc8-1995-4417-8832-26944d93fb49-service-ca\") pod \"console-56946cb5cf-n2zjq\" (UID: \"09914dc8-1995-4417-8832-26944d93fb49\") " pod="openshift-console/console-56946cb5cf-n2zjq"
Sep 15 16:30:31.695565 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:31.695550    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09914dc8-1995-4417-8832-26944d93fb49-trusted-ca-bundle\") pod \"console-56946cb5cf-n2zjq\" (UID: \"09914dc8-1995-4417-8832-26944d93fb49\") " pod="openshift-console/console-56946cb5cf-n2zjq"
Sep 15 16:30:31.695750 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:31.695677    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/09914dc8-1995-4417-8832-26944d93fb49-console-oauth-config\") pod \"console-56946cb5cf-n2zjq\" (UID: \"09914dc8-1995-4417-8832-26944d93fb49\") " pod="openshift-console/console-56946cb5cf-n2zjq"
Sep 15 16:30:31.695750 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:31.695707    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/09914dc8-1995-4417-8832-26944d93fb49-oauth-serving-cert\") pod \"console-56946cb5cf-n2zjq\" (UID: \"09914dc8-1995-4417-8832-26944d93fb49\") " pod="openshift-console/console-56946cb5cf-n2zjq"
Sep 15 16:30:31.696290 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:31.696257    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/09914dc8-1995-4417-8832-26944d93fb49-service-ca\") pod \"console-56946cb5cf-n2zjq\" (UID: \"09914dc8-1995-4417-8832-26944d93fb49\") " pod="openshift-console/console-56946cb5cf-n2zjq"
Sep 15 16:30:31.696374 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:31.696322    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/09914dc8-1995-4417-8832-26944d93fb49-oauth-serving-cert\") pod \"console-56946cb5cf-n2zjq\" (UID: \"09914dc8-1995-4417-8832-26944d93fb49\") " pod="openshift-console/console-56946cb5cf-n2zjq"
Sep 15 16:30:31.696460 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:31.696437    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/09914dc8-1995-4417-8832-26944d93fb49-console-config\") pod \"console-56946cb5cf-n2zjq\" (UID: \"09914dc8-1995-4417-8832-26944d93fb49\") " pod="openshift-console/console-56946cb5cf-n2zjq"
Sep 15 16:30:31.696555 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:31.696536    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09914dc8-1995-4417-8832-26944d93fb49-trusted-ca-bundle\") pod \"console-56946cb5cf-n2zjq\" (UID: \"09914dc8-1995-4417-8832-26944d93fb49\") " pod="openshift-console/console-56946cb5cf-n2zjq"
Sep 15 16:30:31.698153 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:31.698134    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/09914dc8-1995-4417-8832-26944d93fb49-console-serving-cert\") pod \"console-56946cb5cf-n2zjq\" (UID: \"09914dc8-1995-4417-8832-26944d93fb49\") " pod="openshift-console/console-56946cb5cf-n2zjq"
Sep 15 16:30:31.698153 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:31.698144    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/09914dc8-1995-4417-8832-26944d93fb49-console-oauth-config\") pod \"console-56946cb5cf-n2zjq\" (UID: \"09914dc8-1995-4417-8832-26944d93fb49\") " pod="openshift-console/console-56946cb5cf-n2zjq"
Sep 15 16:30:31.703813 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:31.703767    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l2b2\" (UniqueName: \"kubernetes.io/projected/09914dc8-1995-4417-8832-26944d93fb49-kube-api-access-7l2b2\") pod \"console-56946cb5cf-n2zjq\" (UID: \"09914dc8-1995-4417-8832-26944d93fb49\") " pod="openshift-console/console-56946cb5cf-n2zjq"
Sep 15 16:30:31.761493 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:31.761468    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56946cb5cf-n2zjq"
Sep 15 16:30:32.030304 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:32.030228    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56946cb5cf-n2zjq"]
Sep 15 16:30:32.032997 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:30:32.032972    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09914dc8_1995_4417_8832_26944d93fb49.slice/crio-9df77534c02439565ec7c41df6b06f85e95b3ca57a60b653094f7f5241db3a11 WatchSource:0}: Error finding container 9df77534c02439565ec7c41df6b06f85e95b3ca57a60b653094f7f5241db3a11: Status 404 returned error can't find the container with id 9df77534c02439565ec7c41df6b06f85e95b3ca57a60b653094f7f5241db3a11
Sep 15 16:30:32.889750 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:32.889720    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56946cb5cf-n2zjq" event={"ID":"09914dc8-1995-4417-8832-26944d93fb49","Type":"ContainerStarted","Data":"defc6e459b3b78aa1fba3c6054b54f93ae1f934695fb18a0f447be15cdbcb17e"}
Sep 15 16:30:32.890104 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:32.889808    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56946cb5cf-n2zjq" event={"ID":"09914dc8-1995-4417-8832-26944d93fb49","Type":"ContainerStarted","Data":"9df77534c02439565ec7c41df6b06f85e95b3ca57a60b653094f7f5241db3a11"}
Sep 15 16:30:32.907251 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:32.907206    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56946cb5cf-n2zjq" podStartSLOduration=1.907193222 podStartE2EDuration="1.907193222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 16:30:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:30:32.906574916 +0000 UTC m=+249.536229267" watchObservedRunningTime="2026-09-15 16:30:32.907193222 +0000 UTC m=+249.536847573"
Sep 15 16:30:41.762178 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:41.762148    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56946cb5cf-n2zjq"
Sep 15 16:30:41.762543 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:41.762190    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56946cb5cf-n2zjq"
Sep 15 16:30:41.766300 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:41.766282    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56946cb5cf-n2zjq"
Sep 15 16:30:41.923303 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:41.923281    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56946cb5cf-n2zjq"
Sep 15 16:30:41.968577 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:30:41.968550    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7fb8b847f-m8g5t"]
Sep 15 16:31:06.986861 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:06.986803    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7fb8b847f-m8g5t" podUID="d043aadb-a85e-4e22-b8fc-903d7ad472db" containerName="console" containerID="cri-o://2ce37e240d23bba2476412e8000f45374c1a775499460c0eeaadc2113995046b" gracePeriod=15
Sep 15 16:31:07.221046 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:07.221026    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fb8b847f-m8g5t"
Sep 15 16:31:07.329201 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:07.329147    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d043aadb-a85e-4e22-b8fc-903d7ad472db-console-oauth-config\") pod \"d043aadb-a85e-4e22-b8fc-903d7ad472db\" (UID: \"d043aadb-a85e-4e22-b8fc-903d7ad472db\") "
Sep 15 16:31:07.329201 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:07.329187    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d043aadb-a85e-4e22-b8fc-903d7ad472db-oauth-serving-cert\") pod \"d043aadb-a85e-4e22-b8fc-903d7ad472db\" (UID: \"d043aadb-a85e-4e22-b8fc-903d7ad472db\") "
Sep 15 16:31:07.329371 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:07.329220    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d043aadb-a85e-4e22-b8fc-903d7ad472db-trusted-ca-bundle\") pod \"d043aadb-a85e-4e22-b8fc-903d7ad472db\" (UID: \"d043aadb-a85e-4e22-b8fc-903d7ad472db\") "
Sep 15 16:31:07.329371 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:07.329240    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d043aadb-a85e-4e22-b8fc-903d7ad472db-console-config\") pod \"d043aadb-a85e-4e22-b8fc-903d7ad472db\" (UID: \"d043aadb-a85e-4e22-b8fc-903d7ad472db\") "
Sep 15 16:31:07.329371 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:07.329255    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d043aadb-a85e-4e22-b8fc-903d7ad472db-service-ca\") pod \"d043aadb-a85e-4e22-b8fc-903d7ad472db\" (UID: \"d043aadb-a85e-4e22-b8fc-903d7ad472db\") "
Sep 15 16:31:07.329525 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:07.329376    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d043aadb-a85e-4e22-b8fc-903d7ad472db-console-serving-cert\") pod \"d043aadb-a85e-4e22-b8fc-903d7ad472db\" (UID: \"d043aadb-a85e-4e22-b8fc-903d7ad472db\") "
Sep 15 16:31:07.329525 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:07.329414    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qv7mg\" (UniqueName: \"kubernetes.io/projected/d043aadb-a85e-4e22-b8fc-903d7ad472db-kube-api-access-qv7mg\") pod \"d043aadb-a85e-4e22-b8fc-903d7ad472db\" (UID: \"d043aadb-a85e-4e22-b8fc-903d7ad472db\") "
Sep 15 16:31:07.329649 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:07.329606    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d043aadb-a85e-4e22-b8fc-903d7ad472db-service-ca" (OuterVolumeSpecName: "service-ca") pod "d043aadb-a85e-4e22-b8fc-903d7ad472db" (UID: "d043aadb-a85e-4e22-b8fc-903d7ad472db"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:31:07.329649 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:07.329616    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d043aadb-a85e-4e22-b8fc-903d7ad472db-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d043aadb-a85e-4e22-b8fc-903d7ad472db" (UID: "d043aadb-a85e-4e22-b8fc-903d7ad472db"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:31:07.329649 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:07.329627    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d043aadb-a85e-4e22-b8fc-903d7ad472db-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d043aadb-a85e-4e22-b8fc-903d7ad472db" (UID: "d043aadb-a85e-4e22-b8fc-903d7ad472db"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:31:07.329649 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:07.329643    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d043aadb-a85e-4e22-b8fc-903d7ad472db-console-config" (OuterVolumeSpecName: "console-config") pod "d043aadb-a85e-4e22-b8fc-903d7ad472db" (UID: "d043aadb-a85e-4e22-b8fc-903d7ad472db"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:31:07.331501 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:07.331481    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d043aadb-a85e-4e22-b8fc-903d7ad472db-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d043aadb-a85e-4e22-b8fc-903d7ad472db" (UID: "d043aadb-a85e-4e22-b8fc-903d7ad472db"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 16:31:07.331582 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:07.331519    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d043aadb-a85e-4e22-b8fc-903d7ad472db-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d043aadb-a85e-4e22-b8fc-903d7ad472db" (UID: "d043aadb-a85e-4e22-b8fc-903d7ad472db"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 16:31:07.331582 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:07.331538    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d043aadb-a85e-4e22-b8fc-903d7ad472db-kube-api-access-qv7mg" (OuterVolumeSpecName: "kube-api-access-qv7mg") pod "d043aadb-a85e-4e22-b8fc-903d7ad472db" (UID: "d043aadb-a85e-4e22-b8fc-903d7ad472db"). InnerVolumeSpecName "kube-api-access-qv7mg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 16:31:07.430815 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:07.430774    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d043aadb-a85e-4e22-b8fc-903d7ad472db-console-oauth-config\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:31:07.430815 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:07.430811    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d043aadb-a85e-4e22-b8fc-903d7ad472db-oauth-serving-cert\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:31:07.430928 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:07.430820    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d043aadb-a85e-4e22-b8fc-903d7ad472db-trusted-ca-bundle\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:31:07.430928 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:07.430828    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d043aadb-a85e-4e22-b8fc-903d7ad472db-console-config\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:31:07.430928 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:07.430836    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d043aadb-a85e-4e22-b8fc-903d7ad472db-service-ca\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:31:07.430928 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:07.430844    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d043aadb-a85e-4e22-b8fc-903d7ad472db-console-serving-cert\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:31:07.430928 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:07.430852    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qv7mg\" (UniqueName: \"kubernetes.io/projected/d043aadb-a85e-4e22-b8fc-903d7ad472db-kube-api-access-qv7mg\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:31:08.011425 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:08.011404    2579 generic.go:356] "Generic (PLEG): container finished" podID="d043aadb-a85e-4e22-b8fc-903d7ad472db" containerID="2ce37e240d23bba2476412e8000f45374c1a775499460c0eeaadc2113995046b" exitCode=0
Sep 15 16:31:08.011681 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:08.011431    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fb8b847f-m8g5t" event={"ID":"d043aadb-a85e-4e22-b8fc-903d7ad472db","Type":"ContainerDied","Data":"2ce37e240d23bba2476412e8000f45374c1a775499460c0eeaadc2113995046b"}
Sep 15 16:31:08.011681 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:08.011448    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fb8b847f-m8g5t"
Sep 15 16:31:08.011681 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:08.011459    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fb8b847f-m8g5t" event={"ID":"d043aadb-a85e-4e22-b8fc-903d7ad472db","Type":"ContainerDied","Data":"07510da6e3104754f0f5c0681ecfa6bf6da312fdcdfc2bac60a85a280e40b110"}
Sep 15 16:31:08.011681 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:08.011478    2579 scope.go:117] "RemoveContainer" containerID="2ce37e240d23bba2476412e8000f45374c1a775499460c0eeaadc2113995046b"
Sep 15 16:31:08.019332 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:08.019309    2579 scope.go:117] "RemoveContainer" containerID="2ce37e240d23bba2476412e8000f45374c1a775499460c0eeaadc2113995046b"
Sep 15 16:31:08.019594 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:31:08.019576    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ce37e240d23bba2476412e8000f45374c1a775499460c0eeaadc2113995046b\": container with ID starting with 2ce37e240d23bba2476412e8000f45374c1a775499460c0eeaadc2113995046b not found: ID does not exist" containerID="2ce37e240d23bba2476412e8000f45374c1a775499460c0eeaadc2113995046b"
Sep 15 16:31:08.019649 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:08.019606    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ce37e240d23bba2476412e8000f45374c1a775499460c0eeaadc2113995046b"} err="failed to get container status \"2ce37e240d23bba2476412e8000f45374c1a775499460c0eeaadc2113995046b\": rpc error: code = NotFound desc = could not find container \"2ce37e240d23bba2476412e8000f45374c1a775499460c0eeaadc2113995046b\": container with ID starting with 2ce37e240d23bba2476412e8000f45374c1a775499460c0eeaadc2113995046b not found: ID does not exist"
Sep 15 16:31:08.027137 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:08.027117    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7fb8b847f-m8g5t"]
Sep 15 16:31:08.030467 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:08.030449    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7fb8b847f-m8g5t"]
Sep 15 16:31:09.907280 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:09.907251    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d043aadb-a85e-4e22-b8fc-903d7ad472db" path="/var/lib/kubelet/pods/d043aadb-a85e-4e22-b8fc-903d7ad472db/volumes"
Sep 15 16:31:23.798399 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:23.798376    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4rdn_44ca88f0-c149-443e-a2d1-a30964775818/ovn-acl-logging/0.log"
Sep 15 16:31:23.800966 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:23.800936    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4rdn_44ca88f0-c149-443e-a2d1-a30964775818/ovn-acl-logging/0.log"
Sep 15 16:31:23.803803 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:31:23.803770    2579 kubelet.go:1648] "Image garbage collection succeeded"
Sep 15 16:33:01.419485 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:01.419463    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75667bb54d-bw6pp"]
Sep 15 16:33:01.419912 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:01.419731    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d043aadb-a85e-4e22-b8fc-903d7ad472db" containerName="console"
Sep 15 16:33:01.419912 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:01.419741    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d043aadb-a85e-4e22-b8fc-903d7ad472db" containerName="console"
Sep 15 16:33:01.419912 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:01.419811    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d043aadb-a85e-4e22-b8fc-903d7ad472db" containerName="console"
Sep 15 16:33:01.422542 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:01.422528    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75667bb54d-bw6pp"
Sep 15 16:33:01.442747 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:01.442728    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75667bb54d-bw6pp"]
Sep 15 16:33:01.500527 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:01.500500    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbl27\" (UniqueName: \"kubernetes.io/projected/07f7ed38-cd70-409b-a08e-5449c1829b42-kube-api-access-mbl27\") pod \"console-75667bb54d-bw6pp\" (UID: \"07f7ed38-cd70-409b-a08e-5449c1829b42\") " pod="openshift-console/console-75667bb54d-bw6pp"
Sep 15 16:33:01.500644 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:01.500543    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07f7ed38-cd70-409b-a08e-5449c1829b42-service-ca\") pod \"console-75667bb54d-bw6pp\" (UID: \"07f7ed38-cd70-409b-a08e-5449c1829b42\") " pod="openshift-console/console-75667bb54d-bw6pp"
Sep 15 16:33:01.500644 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:01.500560    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07f7ed38-cd70-409b-a08e-5449c1829b42-trusted-ca-bundle\") pod \"console-75667bb54d-bw6pp\" (UID: \"07f7ed38-cd70-409b-a08e-5449c1829b42\") " pod="openshift-console/console-75667bb54d-bw6pp"
Sep 15 16:33:01.500644 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:01.500577    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07f7ed38-cd70-409b-a08e-5449c1829b42-console-serving-cert\") pod \"console-75667bb54d-bw6pp\" (UID: \"07f7ed38-cd70-409b-a08e-5449c1829b42\") " pod="openshift-console/console-75667bb54d-bw6pp"
Sep 15 16:33:01.500644 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:01.500593    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07f7ed38-cd70-409b-a08e-5449c1829b42-console-config\") pod \"console-75667bb54d-bw6pp\" (UID: \"07f7ed38-cd70-409b-a08e-5449c1829b42\") " pod="openshift-console/console-75667bb54d-bw6pp"
Sep 15 16:33:01.500644 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:01.500614    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07f7ed38-cd70-409b-a08e-5449c1829b42-oauth-serving-cert\") pod \"console-75667bb54d-bw6pp\" (UID: \"07f7ed38-cd70-409b-a08e-5449c1829b42\") " pod="openshift-console/console-75667bb54d-bw6pp"
Sep 15 16:33:01.500644 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:01.500639    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07f7ed38-cd70-409b-a08e-5449c1829b42-console-oauth-config\") pod \"console-75667bb54d-bw6pp\" (UID: \"07f7ed38-cd70-409b-a08e-5449c1829b42\") " pod="openshift-console/console-75667bb54d-bw6pp"
Sep 15 16:33:01.600965 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:01.600938    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07f7ed38-cd70-409b-a08e-5449c1829b42-service-ca\") pod \"console-75667bb54d-bw6pp\" (UID: \"07f7ed38-cd70-409b-a08e-5449c1829b42\") " pod="openshift-console/console-75667bb54d-bw6pp"
Sep 15 16:33:01.601057 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:01.600970    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07f7ed38-cd70-409b-a08e-5449c1829b42-trusted-ca-bundle\") pod \"console-75667bb54d-bw6pp\" (UID: \"07f7ed38-cd70-409b-a08e-5449c1829b42\") " pod="openshift-console/console-75667bb54d-bw6pp"
Sep 15 16:33:01.601057 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:01.600990    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07f7ed38-cd70-409b-a08e-5449c1829b42-console-serving-cert\") pod \"console-75667bb54d-bw6pp\" (UID: \"07f7ed38-cd70-409b-a08e-5449c1829b42\") " pod="openshift-console/console-75667bb54d-bw6pp"
Sep 15 16:33:01.601057 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:01.601003    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07f7ed38-cd70-409b-a08e-5449c1829b42-console-config\") pod \"console-75667bb54d-bw6pp\" (UID: \"07f7ed38-cd70-409b-a08e-5449c1829b42\") " pod="openshift-console/console-75667bb54d-bw6pp"
Sep 15 16:33:01.601057 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:01.601028    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07f7ed38-cd70-409b-a08e-5449c1829b42-oauth-serving-cert\") pod \"console-75667bb54d-bw6pp\" (UID: \"07f7ed38-cd70-409b-a08e-5449c1829b42\") " pod="openshift-console/console-75667bb54d-bw6pp"
Sep 15 16:33:01.601251 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:01.601063    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07f7ed38-cd70-409b-a08e-5449c1829b42-console-oauth-config\") pod \"console-75667bb54d-bw6pp\" (UID: \"07f7ed38-cd70-409b-a08e-5449c1829b42\") " pod="openshift-console/console-75667bb54d-bw6pp"
Sep 15 16:33:01.601251 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:01.601114    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbl27\" (UniqueName: \"kubernetes.io/projected/07f7ed38-cd70-409b-a08e-5449c1829b42-kube-api-access-mbl27\") pod \"console-75667bb54d-bw6pp\" (UID: \"07f7ed38-cd70-409b-a08e-5449c1829b42\") " pod="openshift-console/console-75667bb54d-bw6pp"
Sep 15 16:33:01.601680 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:01.601643    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07f7ed38-cd70-409b-a08e-5449c1829b42-service-ca\") pod \"console-75667bb54d-bw6pp\" (UID: \"07f7ed38-cd70-409b-a08e-5449c1829b42\") " pod="openshift-console/console-75667bb54d-bw6pp"
Sep 15 16:33:01.601770 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:01.601751    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07f7ed38-cd70-409b-a08e-5449c1829b42-console-config\") pod \"console-75667bb54d-bw6pp\" (UID: \"07f7ed38-cd70-409b-a08e-5449c1829b42\") " pod="openshift-console/console-75667bb54d-bw6pp"
Sep 15 16:33:01.602054 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:01.601871    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07f7ed38-cd70-409b-a08e-5449c1829b42-trusted-ca-bundle\") pod \"console-75667bb54d-bw6pp\" (UID: \"07f7ed38-cd70-409b-a08e-5449c1829b42\") " pod="openshift-console/console-75667bb54d-bw6pp"
Sep 15 16:33:01.602054 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:01.601885    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07f7ed38-cd70-409b-a08e-5449c1829b42-oauth-serving-cert\") pod \"console-75667bb54d-bw6pp\" (UID: \"07f7ed38-cd70-409b-a08e-5449c1829b42\") " pod="openshift-console/console-75667bb54d-bw6pp"
Sep 15 16:33:01.603628 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:01.603594    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07f7ed38-cd70-409b-a08e-5449c1829b42-console-serving-cert\") pod \"console-75667bb54d-bw6pp\" (UID: \"07f7ed38-cd70-409b-a08e-5449c1829b42\") " pod="openshift-console/console-75667bb54d-bw6pp"
Sep 15 16:33:01.603755 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:01.603734    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07f7ed38-cd70-409b-a08e-5449c1829b42-console-oauth-config\") pod \"console-75667bb54d-bw6pp\" (UID: \"07f7ed38-cd70-409b-a08e-5449c1829b42\") " pod="openshift-console/console-75667bb54d-bw6pp"
Sep 15 16:33:01.609851 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:01.609830    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbl27\" (UniqueName: \"kubernetes.io/projected/07f7ed38-cd70-409b-a08e-5449c1829b42-kube-api-access-mbl27\") pod \"console-75667bb54d-bw6pp\" (UID: \"07f7ed38-cd70-409b-a08e-5449c1829b42\") " pod="openshift-console/console-75667bb54d-bw6pp"
Sep 15 16:33:01.731212 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:01.731193    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75667bb54d-bw6pp"
Sep 15 16:33:01.999542 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:01.999445    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75667bb54d-bw6pp"]
Sep 15 16:33:02.002036 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:33:02.002011    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07f7ed38_cd70_409b_a08e_5449c1829b42.slice/crio-5558190e025eac63f118a2e65d0a1a863d01ca7762e21083043287e24f07bd21 WatchSource:0}: Error finding container 5558190e025eac63f118a2e65d0a1a863d01ca7762e21083043287e24f07bd21: Status 404 returned error can't find the container with id 5558190e025eac63f118a2e65d0a1a863d01ca7762e21083043287e24f07bd21
Sep 15 16:33:02.415255 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:02.415176    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75667bb54d-bw6pp" event={"ID":"07f7ed38-cd70-409b-a08e-5449c1829b42","Type":"ContainerStarted","Data":"73d1b34380ab08e68f87447aa9d7095ccde38e058301e1e5192460e042c66911"}
Sep 15 16:33:02.415255 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:02.415241    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75667bb54d-bw6pp" event={"ID":"07f7ed38-cd70-409b-a08e-5449c1829b42","Type":"ContainerStarted","Data":"5558190e025eac63f118a2e65d0a1a863d01ca7762e21083043287e24f07bd21"}
Sep 15 16:33:02.434196 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:02.434154    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75667bb54d-bw6pp" podStartSLOduration=1.434140206 podStartE2EDuration="1.434140206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 16:33:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:33:02.433242834 +0000 UTC m=+399.062897185" watchObservedRunningTime="2026-09-15 16:33:02.434140206 +0000 UTC m=+399.063794558"
Sep 15 16:33:11.731711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:11.731670    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-75667bb54d-bw6pp"
Sep 15 16:33:11.732053 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:11.731731    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75667bb54d-bw6pp"
Sep 15 16:33:11.735802 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:11.735766    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-75667bb54d-bw6pp"
Sep 15 16:33:12.452800 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:12.452760    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-75667bb54d-bw6pp"
Sep 15 16:33:12.501633 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:12.501604    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56946cb5cf-n2zjq"]
Sep 15 16:33:37.520482 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:37.520437    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-56946cb5cf-n2zjq" podUID="09914dc8-1995-4417-8832-26944d93fb49" containerName="console" containerID="cri-o://defc6e459b3b78aa1fba3c6054b54f93ae1f934695fb18a0f447be15cdbcb17e" gracePeriod=15
Sep 15 16:33:37.755874 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:37.755853    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56946cb5cf-n2zjq"
Sep 15 16:33:37.929737 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:37.929676    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/09914dc8-1995-4417-8832-26944d93fb49-console-serving-cert\") pod \"09914dc8-1995-4417-8832-26944d93fb49\" (UID: \"09914dc8-1995-4417-8832-26944d93fb49\") "
Sep 15 16:33:37.929881 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:37.929750    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/09914dc8-1995-4417-8832-26944d93fb49-console-config\") pod \"09914dc8-1995-4417-8832-26944d93fb49\" (UID: \"09914dc8-1995-4417-8832-26944d93fb49\") "
Sep 15 16:33:37.929881 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:37.929806    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09914dc8-1995-4417-8832-26944d93fb49-trusted-ca-bundle\") pod \"09914dc8-1995-4417-8832-26944d93fb49\" (UID: \"09914dc8-1995-4417-8832-26944d93fb49\") "
Sep 15 16:33:37.930010 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:37.929885    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/09914dc8-1995-4417-8832-26944d93fb49-oauth-serving-cert\") pod \"09914dc8-1995-4417-8832-26944d93fb49\" (UID: \"09914dc8-1995-4417-8832-26944d93fb49\") "
Sep 15 16:33:37.930010 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:37.929927    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/09914dc8-1995-4417-8832-26944d93fb49-console-oauth-config\") pod \"09914dc8-1995-4417-8832-26944d93fb49\" (UID: \"09914dc8-1995-4417-8832-26944d93fb49\") "
Sep 15 16:33:37.930010 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:37.929965    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/09914dc8-1995-4417-8832-26944d93fb49-service-ca\") pod \"09914dc8-1995-4417-8832-26944d93fb49\" (UID: \"09914dc8-1995-4417-8832-26944d93fb49\") "
Sep 15 16:33:37.930010 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:37.929989    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7l2b2\" (UniqueName: \"kubernetes.io/projected/09914dc8-1995-4417-8832-26944d93fb49-kube-api-access-7l2b2\") pod \"09914dc8-1995-4417-8832-26944d93fb49\" (UID: \"09914dc8-1995-4417-8832-26944d93fb49\") "
Sep 15 16:33:37.930257 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:37.930229    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09914dc8-1995-4417-8832-26944d93fb49-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "09914dc8-1995-4417-8832-26944d93fb49" (UID: "09914dc8-1995-4417-8832-26944d93fb49"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:33:37.930257 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:37.930251    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09914dc8-1995-4417-8832-26944d93fb49-console-config" (OuterVolumeSpecName: "console-config") pod "09914dc8-1995-4417-8832-26944d93fb49" (UID: "09914dc8-1995-4417-8832-26944d93fb49"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:33:37.930362 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:37.930340    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09914dc8-1995-4417-8832-26944d93fb49-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "09914dc8-1995-4417-8832-26944d93fb49" (UID: "09914dc8-1995-4417-8832-26944d93fb49"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:33:37.930425 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:37.930357    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09914dc8-1995-4417-8832-26944d93fb49-service-ca" (OuterVolumeSpecName: "service-ca") pod "09914dc8-1995-4417-8832-26944d93fb49" (UID: "09914dc8-1995-4417-8832-26944d93fb49"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:33:37.932160 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:37.932133    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09914dc8-1995-4417-8832-26944d93fb49-kube-api-access-7l2b2" (OuterVolumeSpecName: "kube-api-access-7l2b2") pod "09914dc8-1995-4417-8832-26944d93fb49" (UID: "09914dc8-1995-4417-8832-26944d93fb49"). InnerVolumeSpecName "kube-api-access-7l2b2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 16:33:37.932160 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:37.932148    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09914dc8-1995-4417-8832-26944d93fb49-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "09914dc8-1995-4417-8832-26944d93fb49" (UID: "09914dc8-1995-4417-8832-26944d93fb49"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 16:33:37.932301 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:37.932216    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09914dc8-1995-4417-8832-26944d93fb49-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "09914dc8-1995-4417-8832-26944d93fb49" (UID: "09914dc8-1995-4417-8832-26944d93fb49"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 16:33:38.031242 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:38.031219    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/09914dc8-1995-4417-8832-26944d93fb49-console-serving-cert\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:33:38.031242 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:38.031241    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/09914dc8-1995-4417-8832-26944d93fb49-console-config\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:33:38.031372 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:38.031255    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09914dc8-1995-4417-8832-26944d93fb49-trusted-ca-bundle\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:33:38.031372 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:38.031266    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/09914dc8-1995-4417-8832-26944d93fb49-oauth-serving-cert\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:33:38.031372 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:38.031277    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/09914dc8-1995-4417-8832-26944d93fb49-console-oauth-config\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:33:38.031372 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:38.031288    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/09914dc8-1995-4417-8832-26944d93fb49-service-ca\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:33:38.031372 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:38.031299    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7l2b2\" (UniqueName: \"kubernetes.io/projected/09914dc8-1995-4417-8832-26944d93fb49-kube-api-access-7l2b2\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:33:38.542694 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:38.542667    2579 generic.go:356] "Generic (PLEG): container finished" podID="09914dc8-1995-4417-8832-26944d93fb49" containerID="defc6e459b3b78aa1fba3c6054b54f93ae1f934695fb18a0f447be15cdbcb17e" exitCode=0
Sep 15 16:33:38.543006 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:38.542704    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56946cb5cf-n2zjq" event={"ID":"09914dc8-1995-4417-8832-26944d93fb49","Type":"ContainerDied","Data":"defc6e459b3b78aa1fba3c6054b54f93ae1f934695fb18a0f447be15cdbcb17e"}
Sep 15 16:33:38.543006 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:38.542708    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56946cb5cf-n2zjq"
Sep 15 16:33:38.543006 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:38.542737    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56946cb5cf-n2zjq" event={"ID":"09914dc8-1995-4417-8832-26944d93fb49","Type":"ContainerDied","Data":"9df77534c02439565ec7c41df6b06f85e95b3ca57a60b653094f7f5241db3a11"}
Sep 15 16:33:38.543006 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:38.542760    2579 scope.go:117] "RemoveContainer" containerID="defc6e459b3b78aa1fba3c6054b54f93ae1f934695fb18a0f447be15cdbcb17e"
Sep 15 16:33:38.550956 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:38.550938    2579 scope.go:117] "RemoveContainer" containerID="defc6e459b3b78aa1fba3c6054b54f93ae1f934695fb18a0f447be15cdbcb17e"
Sep 15 16:33:38.551218 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:33:38.551202    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"defc6e459b3b78aa1fba3c6054b54f93ae1f934695fb18a0f447be15cdbcb17e\": container with ID starting with defc6e459b3b78aa1fba3c6054b54f93ae1f934695fb18a0f447be15cdbcb17e not found: ID does not exist" containerID="defc6e459b3b78aa1fba3c6054b54f93ae1f934695fb18a0f447be15cdbcb17e"
Sep 15 16:33:38.551264 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:38.551226    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"defc6e459b3b78aa1fba3c6054b54f93ae1f934695fb18a0f447be15cdbcb17e"} err="failed to get container status \"defc6e459b3b78aa1fba3c6054b54f93ae1f934695fb18a0f447be15cdbcb17e\": rpc error: code = NotFound desc = could not find container \"defc6e459b3b78aa1fba3c6054b54f93ae1f934695fb18a0f447be15cdbcb17e\": container with ID starting with defc6e459b3b78aa1fba3c6054b54f93ae1f934695fb18a0f447be15cdbcb17e not found: ID does not exist"
Sep 15 16:33:38.572520 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:38.572497    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56946cb5cf-n2zjq"]
Sep 15 16:33:38.584130 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:38.584113    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56946cb5cf-n2zjq"]
Sep 15 16:33:39.908775 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:39.908739    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09914dc8-1995-4417-8832-26944d93fb49" path="/var/lib/kubelet/pods/09914dc8-1995-4417-8832-26944d93fb49/volumes"
Sep 15 16:33:46.044276 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:46.044206    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aggh4g"]
Sep 15 16:33:46.044590 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:46.044462    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09914dc8-1995-4417-8832-26944d93fb49" containerName="console"
Sep 15 16:33:46.044590 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:46.044472    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="09914dc8-1995-4417-8832-26944d93fb49" containerName="console"
Sep 15 16:33:46.044590 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:46.044517    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="09914dc8-1995-4417-8832-26944d93fb49" containerName="console"
Sep 15 16:33:46.048802 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:46.048772    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aggh4g"
Sep 15 16:33:46.052615 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:46.052593    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bn6jg\""
Sep 15 16:33:46.052723 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:46.052595    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 15 16:33:46.053147 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:46.053132    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 15 16:33:46.075826 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:46.075796    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aggh4g"]
Sep 15 16:33:46.184432 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:46.184406    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pgks\" (UniqueName: \"kubernetes.io/projected/5339bb05-4286-4444-b349-13a31f9c3991-kube-api-access-2pgks\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aggh4g\" (UID: \"5339bb05-4286-4444-b349-13a31f9c3991\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aggh4g"
Sep 15 16:33:46.184548 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:46.184476    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5339bb05-4286-4444-b349-13a31f9c3991-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aggh4g\" (UID: \"5339bb05-4286-4444-b349-13a31f9c3991\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aggh4g"
Sep 15 16:33:46.184548 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:46.184519    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5339bb05-4286-4444-b349-13a31f9c3991-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aggh4g\" (UID: \"5339bb05-4286-4444-b349-13a31f9c3991\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aggh4g"
Sep 15 16:33:46.284889 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:46.284864    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5339bb05-4286-4444-b349-13a31f9c3991-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aggh4g\" (UID: \"5339bb05-4286-4444-b349-13a31f9c3991\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aggh4g"
Sep 15 16:33:46.284981 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:46.284894    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pgks\" (UniqueName: \"kubernetes.io/projected/5339bb05-4286-4444-b349-13a31f9c3991-kube-api-access-2pgks\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aggh4g\" (UID: \"5339bb05-4286-4444-b349-13a31f9c3991\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aggh4g"
Sep 15 16:33:46.285048 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:46.285025    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5339bb05-4286-4444-b349-13a31f9c3991-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aggh4g\" (UID: \"5339bb05-4286-4444-b349-13a31f9c3991\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aggh4g"
Sep 15 16:33:46.285238 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:46.285223    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5339bb05-4286-4444-b349-13a31f9c3991-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aggh4g\" (UID: \"5339bb05-4286-4444-b349-13a31f9c3991\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aggh4g"
Sep 15 16:33:46.285285 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:46.285269    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5339bb05-4286-4444-b349-13a31f9c3991-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aggh4g\" (UID: \"5339bb05-4286-4444-b349-13a31f9c3991\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aggh4g"
Sep 15 16:33:46.301454 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:46.301406    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pgks\" (UniqueName: \"kubernetes.io/projected/5339bb05-4286-4444-b349-13a31f9c3991-kube-api-access-2pgks\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aggh4g\" (UID: \"5339bb05-4286-4444-b349-13a31f9c3991\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aggh4g"
Sep 15 16:33:46.356857 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:46.356836    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aggh4g"
Sep 15 16:33:46.623128 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:46.623070    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aggh4g"]
Sep 15 16:33:46.625272 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:33:46.625240    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5339bb05_4286_4444_b349_13a31f9c3991.slice/crio-20b878b72ab7549528e729d69d758c900979cd9f8eec602fd7343d9c7c91dfc1 WatchSource:0}: Error finding container 20b878b72ab7549528e729d69d758c900979cd9f8eec602fd7343d9c7c91dfc1: Status 404 returned error can't find the container with id 20b878b72ab7549528e729d69d758c900979cd9f8eec602fd7343d9c7c91dfc1
Sep 15 16:33:46.627258 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:46.627241    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 16:33:47.574597 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:47.574564    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aggh4g" event={"ID":"5339bb05-4286-4444-b349-13a31f9c3991","Type":"ContainerStarted","Data":"20b878b72ab7549528e729d69d758c900979cd9f8eec602fd7343d9c7c91dfc1"}
Sep 15 16:33:53.600938 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:53.600908    2579 generic.go:356] "Generic (PLEG): container finished" podID="5339bb05-4286-4444-b349-13a31f9c3991" containerID="5501f6d31e7d940ff55423a54bdee040d8b8cf4e64be89a711a5d9e901764487" exitCode=0
Sep 15 16:33:53.601215 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:53.600995    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aggh4g" event={"ID":"5339bb05-4286-4444-b349-13a31f9c3991","Type":"ContainerDied","Data":"5501f6d31e7d940ff55423a54bdee040d8b8cf4e64be89a711a5d9e901764487"}
Sep 15 16:33:57.618297 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:57.618269    2579 generic.go:356] "Generic (PLEG): container finished" podID="5339bb05-4286-4444-b349-13a31f9c3991" containerID="a3d323b2baeb94f10ac67e907dc6b7dc4cf33460408092d4b8ed377e6ae1c3b7" exitCode=0
Sep 15 16:33:57.618632 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:33:57.618349    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aggh4g" event={"ID":"5339bb05-4286-4444-b349-13a31f9c3991","Type":"ContainerDied","Data":"a3d323b2baeb94f10ac67e907dc6b7dc4cf33460408092d4b8ed377e6ae1c3b7"}
Sep 15 16:34:03.644796 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:03.644712    2579 generic.go:356] "Generic (PLEG): container finished" podID="5339bb05-4286-4444-b349-13a31f9c3991" containerID="6255cb0db3f450df651db3a72cd932b9e8fd0bcc2bff17f8dd1165aee18cbab0" exitCode=0
Sep 15 16:34:03.645125 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:03.644808    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aggh4g" event={"ID":"5339bb05-4286-4444-b349-13a31f9c3991","Type":"ContainerDied","Data":"6255cb0db3f450df651db3a72cd932b9e8fd0bcc2bff17f8dd1165aee18cbab0"}
Sep 15 16:34:04.769847 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:04.769826    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aggh4g"
Sep 15 16:34:04.822044 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:04.822021    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2pgks\" (UniqueName: \"kubernetes.io/projected/5339bb05-4286-4444-b349-13a31f9c3991-kube-api-access-2pgks\") pod \"5339bb05-4286-4444-b349-13a31f9c3991\" (UID: \"5339bb05-4286-4444-b349-13a31f9c3991\") "
Sep 15 16:34:04.822138 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:04.822073    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5339bb05-4286-4444-b349-13a31f9c3991-bundle\") pod \"5339bb05-4286-4444-b349-13a31f9c3991\" (UID: \"5339bb05-4286-4444-b349-13a31f9c3991\") "
Sep 15 16:34:04.822138 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:04.822111    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5339bb05-4286-4444-b349-13a31f9c3991-util\") pod \"5339bb05-4286-4444-b349-13a31f9c3991\" (UID: \"5339bb05-4286-4444-b349-13a31f9c3991\") "
Sep 15 16:34:04.822722 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:04.822699    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5339bb05-4286-4444-b349-13a31f9c3991-bundle" (OuterVolumeSpecName: "bundle") pod "5339bb05-4286-4444-b349-13a31f9c3991" (UID: "5339bb05-4286-4444-b349-13a31f9c3991"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 16:34:04.824064 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:04.824044    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5339bb05-4286-4444-b349-13a31f9c3991-kube-api-access-2pgks" (OuterVolumeSpecName: "kube-api-access-2pgks") pod "5339bb05-4286-4444-b349-13a31f9c3991" (UID: "5339bb05-4286-4444-b349-13a31f9c3991"). InnerVolumeSpecName "kube-api-access-2pgks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 16:34:04.826216 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:04.826192    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5339bb05-4286-4444-b349-13a31f9c3991-util" (OuterVolumeSpecName: "util") pod "5339bb05-4286-4444-b349-13a31f9c3991" (UID: "5339bb05-4286-4444-b349-13a31f9c3991"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 16:34:04.922537 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:04.922480    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5339bb05-4286-4444-b349-13a31f9c3991-util\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:34:04.922537 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:04.922504    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pgks\" (UniqueName: \"kubernetes.io/projected/5339bb05-4286-4444-b349-13a31f9c3991-kube-api-access-2pgks\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:34:04.922537 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:04.922519    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5339bb05-4286-4444-b349-13a31f9c3991-bundle\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:34:05.653310 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:05.653287    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aggh4g" event={"ID":"5339bb05-4286-4444-b349-13a31f9c3991","Type":"ContainerDied","Data":"20b878b72ab7549528e729d69d758c900979cd9f8eec602fd7343d9c7c91dfc1"}
Sep 15 16:34:05.653386 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:05.653320    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="20b878b72ab7549528e729d69d758c900979cd9f8eec602fd7343d9c7c91dfc1"
Sep 15 16:34:05.653386 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:05.653323    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1aggh4g"
Sep 15 16:34:08.302056 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:08.302027    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-hjdpd"]
Sep 15 16:34:08.302477 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:08.302292    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5339bb05-4286-4444-b349-13a31f9c3991" containerName="pull"
Sep 15 16:34:08.302477 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:08.302303    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5339bb05-4286-4444-b349-13a31f9c3991" containerName="pull"
Sep 15 16:34:08.302477 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:08.302311    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5339bb05-4286-4444-b349-13a31f9c3991" containerName="util"
Sep 15 16:34:08.302477 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:08.302316    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5339bb05-4286-4444-b349-13a31f9c3991" containerName="util"
Sep 15 16:34:08.302477 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:08.302322    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5339bb05-4286-4444-b349-13a31f9c3991" containerName="extract"
Sep 15 16:34:08.302477 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:08.302327    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5339bb05-4286-4444-b349-13a31f9c3991" containerName="extract"
Sep 15 16:34:08.302477 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:08.302372    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5339bb05-4286-4444-b349-13a31f9c3991" containerName="extract"
Sep 15 16:34:08.304966 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:08.304951    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-hjdpd"
Sep 15 16:34:08.307360 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:08.307334    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Sep 15 16:34:08.307506 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:08.307370    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Sep 15 16:34:08.307506 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:08.307488    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-smdtp\""
Sep 15 16:34:08.307621 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:08.307572    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Sep 15 16:34:08.314640 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:08.314621    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-hjdpd"]
Sep 15 16:34:08.343083 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:08.343057    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqtwz\" (UniqueName: \"kubernetes.io/projected/48430970-aea9-403b-b7da-04f418df5721-kube-api-access-kqtwz\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-hjdpd\" (UID: \"48430970-aea9-403b-b7da-04f418df5721\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-hjdpd"
Sep 15 16:34:08.343176 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:08.343109    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/48430970-aea9-403b-b7da-04f418df5721-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-hjdpd\" (UID: \"48430970-aea9-403b-b7da-04f418df5721\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-hjdpd"
Sep 15 16:34:08.443707 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:08.443682    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqtwz\" (UniqueName: \"kubernetes.io/projected/48430970-aea9-403b-b7da-04f418df5721-kube-api-access-kqtwz\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-hjdpd\" (UID: \"48430970-aea9-403b-b7da-04f418df5721\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-hjdpd"
Sep 15 16:34:08.443820 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:08.443729    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/48430970-aea9-403b-b7da-04f418df5721-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-hjdpd\" (UID: \"48430970-aea9-403b-b7da-04f418df5721\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-hjdpd"
Sep 15 16:34:08.446145 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:08.446125    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/48430970-aea9-403b-b7da-04f418df5721-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-hjdpd\" (UID: \"48430970-aea9-403b-b7da-04f418df5721\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-hjdpd"
Sep 15 16:34:08.453047 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:08.453029    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqtwz\" (UniqueName: \"kubernetes.io/projected/48430970-aea9-403b-b7da-04f418df5721-kube-api-access-kqtwz\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-hjdpd\" (UID: \"48430970-aea9-403b-b7da-04f418df5721\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-hjdpd"
Sep 15 16:34:08.615228 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:08.615182    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-hjdpd"
Sep 15 16:34:08.890124 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:08.890096    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-hjdpd"]
Sep 15 16:34:08.891849 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:34:08.891820    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48430970_aea9_403b_b7da_04f418df5721.slice/crio-28591f90c2061ceb588b1416ab76e0520961bbe5b54f51b4499a9a20744e5dda WatchSource:0}: Error finding container 28591f90c2061ceb588b1416ab76e0520961bbe5b54f51b4499a9a20744e5dda: Status 404 returned error can't find the container with id 28591f90c2061ceb588b1416ab76e0520961bbe5b54f51b4499a9a20744e5dda
Sep 15 16:34:09.670965 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:09.670932    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-hjdpd" event={"ID":"48430970-aea9-403b-b7da-04f418df5721","Type":"ContainerStarted","Data":"28591f90c2061ceb588b1416ab76e0520961bbe5b54f51b4499a9a20744e5dda"}
Sep 15 16:34:13.693924 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:13.693887    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-hjdpd" event={"ID":"48430970-aea9-403b-b7da-04f418df5721","Type":"ContainerStarted","Data":"9dd641c0652240c0255436ad978df1ed81fdfb7149eac3bc0dc1d550d1ae3036"}
Sep 15 16:34:13.694345 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:13.694038    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-hjdpd"
Sep 15 16:34:13.724298 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:13.724259    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-hjdpd" podStartSLOduration=1.686501529 podStartE2EDuration="5.724247052s" totalImagesPullingTime="4.037745523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 16:34:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:34:13.722504025 +0000 UTC m=+470.352158377" watchObservedRunningTime="2026-09-15 16:34:13.724247052 +0000 UTC m=+470.353901402"
Sep 15 16:34:13.782499 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:13.782480    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-579c8958f7-7ns65"]
Sep 15 16:34:13.803971 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:13.803949    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-579c8958f7-7ns65"]
Sep 15 16:34:13.804070 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:13.804011    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-579c8958f7-7ns65"
Sep 15 16:34:13.806827 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:13.806807    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Sep 15 16:34:13.807144 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:13.807122    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Sep 15 16:34:13.807375 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:13.807361    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-xb45x\""
Sep 15 16:34:13.887220 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:13.887196    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/e725bb1a-676c-4091-9129-924b1d3440d1-cabundle0\") pod \"keda-operator-579c8958f7-7ns65\" (UID: \"e725bb1a-676c-4091-9129-924b1d3440d1\") " pod="openshift-keda/keda-operator-579c8958f7-7ns65"
Sep 15 16:34:13.887308 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:13.887222    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znkkt\" (UniqueName: \"kubernetes.io/projected/e725bb1a-676c-4091-9129-924b1d3440d1-kube-api-access-znkkt\") pod \"keda-operator-579c8958f7-7ns65\" (UID: \"e725bb1a-676c-4091-9129-924b1d3440d1\") " pod="openshift-keda/keda-operator-579c8958f7-7ns65"
Sep 15 16:34:13.887308 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:13.887252    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e725bb1a-676c-4091-9129-924b1d3440d1-certificates\") pod \"keda-operator-579c8958f7-7ns65\" (UID: \"e725bb1a-676c-4091-9129-924b1d3440d1\") " pod="openshift-keda/keda-operator-579c8958f7-7ns65"
Sep 15 16:34:13.987742 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:13.987722    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e725bb1a-676c-4091-9129-924b1d3440d1-certificates\") pod \"keda-operator-579c8958f7-7ns65\" (UID: \"e725bb1a-676c-4091-9129-924b1d3440d1\") " pod="openshift-keda/keda-operator-579c8958f7-7ns65"
Sep 15 16:34:13.987869 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:13.987772    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/e725bb1a-676c-4091-9129-924b1d3440d1-cabundle0\") pod \"keda-operator-579c8958f7-7ns65\" (UID: \"e725bb1a-676c-4091-9129-924b1d3440d1\") " pod="openshift-keda/keda-operator-579c8958f7-7ns65"
Sep 15 16:34:13.987869 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:34:13.987862    2579 secret.go:281] references non-existent secret key: ca.crt
Sep 15 16:34:13.987945 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:34:13.987877    2579 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Sep 15 16:34:13.987945 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:34:13.987885    2579 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-579c8958f7-7ns65: references non-existent secret key: ca.crt
Sep 15 16:34:13.987945 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:13.987886    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znkkt\" (UniqueName: \"kubernetes.io/projected/e725bb1a-676c-4091-9129-924b1d3440d1-kube-api-access-znkkt\") pod \"keda-operator-579c8958f7-7ns65\" (UID: \"e725bb1a-676c-4091-9129-924b1d3440d1\") " pod="openshift-keda/keda-operator-579c8958f7-7ns65"
Sep 15 16:34:13.987945 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:34:13.987932    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e725bb1a-676c-4091-9129-924b1d3440d1-certificates podName:e725bb1a-676c-4091-9129-924b1d3440d1 nodeName:}" failed. No retries permitted until 2026-09-15 16:34:14.48791857 +0000 UTC m=+471.117572900 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/e725bb1a-676c-4091-9129-924b1d3440d1-certificates") pod "keda-operator-579c8958f7-7ns65" (UID: "e725bb1a-676c-4091-9129-924b1d3440d1") : references non-existent secret key: ca.crt
Sep 15 16:34:13.988276 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:13.988260    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/e725bb1a-676c-4091-9129-924b1d3440d1-cabundle0\") pod \"keda-operator-579c8958f7-7ns65\" (UID: \"e725bb1a-676c-4091-9129-924b1d3440d1\") " pod="openshift-keda/keda-operator-579c8958f7-7ns65"
Sep 15 16:34:14.030683 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:14.030661    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znkkt\" (UniqueName: \"kubernetes.io/projected/e725bb1a-676c-4091-9129-924b1d3440d1-kube-api-access-znkkt\") pod \"keda-operator-579c8958f7-7ns65\" (UID: \"e725bb1a-676c-4091-9129-924b1d3440d1\") " pod="openshift-keda/keda-operator-579c8958f7-7ns65"
Sep 15 16:34:14.268441 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:14.268389    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-slq96"]
Sep 15 16:34:14.294405 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:14.294381    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-slq96"]
Sep 15 16:34:14.294524 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:14.294465    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-slq96"
Sep 15 16:34:14.296974 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:14.296957    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Sep 15 16:34:14.391256 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:14.391233    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/80dc5400-d63b-4309-82cb-655318fda275-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-slq96\" (UID: \"80dc5400-d63b-4309-82cb-655318fda275\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-slq96"
Sep 15 16:34:14.391337 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:14.391264    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/80dc5400-d63b-4309-82cb-655318fda275-certificates\") pod \"keda-metrics-apiserver-77857b7b98-slq96\" (UID: \"80dc5400-d63b-4309-82cb-655318fda275\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-slq96"
Sep 15 16:34:14.391337 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:14.391278    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7v5f\" (UniqueName: \"kubernetes.io/projected/80dc5400-d63b-4309-82cb-655318fda275-kube-api-access-m7v5f\") pod \"keda-metrics-apiserver-77857b7b98-slq96\" (UID: \"80dc5400-d63b-4309-82cb-655318fda275\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-slq96"
Sep 15 16:34:14.491938 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:14.491918    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/80dc5400-d63b-4309-82cb-655318fda275-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-slq96\" (UID: \"80dc5400-d63b-4309-82cb-655318fda275\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-slq96"
Sep 15 16:34:14.492017 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:14.491948    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/80dc5400-d63b-4309-82cb-655318fda275-certificates\") pod \"keda-metrics-apiserver-77857b7b98-slq96\" (UID: \"80dc5400-d63b-4309-82cb-655318fda275\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-slq96"
Sep 15 16:34:14.492017 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:14.491964    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7v5f\" (UniqueName: \"kubernetes.io/projected/80dc5400-d63b-4309-82cb-655318fda275-kube-api-access-m7v5f\") pod \"keda-metrics-apiserver-77857b7b98-slq96\" (UID: \"80dc5400-d63b-4309-82cb-655318fda275\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-slq96"
Sep 15 16:34:14.492017 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:14.492012    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e725bb1a-676c-4091-9129-924b1d3440d1-certificates\") pod \"keda-operator-579c8958f7-7ns65\" (UID: \"e725bb1a-676c-4091-9129-924b1d3440d1\") " pod="openshift-keda/keda-operator-579c8958f7-7ns65"
Sep 15 16:34:14.492133 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:34:14.492053    2579 secret.go:281] references non-existent secret key: tls.crt
Sep 15 16:34:14.492133 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:34:14.492076    2579 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Sep 15 16:34:14.492133 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:34:14.492091    2579 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-77857b7b98-slq96: references non-existent secret key: tls.crt
Sep 15 16:34:14.492133 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:34:14.492110    2579 secret.go:281] references non-existent secret key: ca.crt
Sep 15 16:34:14.492133 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:34:14.492125    2579 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Sep 15 16:34:14.492133 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:34:14.492134    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/80dc5400-d63b-4309-82cb-655318fda275-certificates podName:80dc5400-d63b-4309-82cb-655318fda275 nodeName:}" failed. No retries permitted until 2026-09-15 16:34:14.992120531 +0000 UTC m=+471.621774860 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/80dc5400-d63b-4309-82cb-655318fda275-certificates") pod "keda-metrics-apiserver-77857b7b98-slq96" (UID: "80dc5400-d63b-4309-82cb-655318fda275") : references non-existent secret key: tls.crt
Sep 15 16:34:14.492339 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:34:14.492135    2579 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-579c8958f7-7ns65: references non-existent secret key: ca.crt
Sep 15 16:34:14.492339 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:34:14.492191    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e725bb1a-676c-4091-9129-924b1d3440d1-certificates podName:e725bb1a-676c-4091-9129-924b1d3440d1 nodeName:}" failed. No retries permitted until 2026-09-15 16:34:15.492175717 +0000 UTC m=+472.121830050 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/e725bb1a-676c-4091-9129-924b1d3440d1-certificates") pod "keda-operator-579c8958f7-7ns65" (UID: "e725bb1a-676c-4091-9129-924b1d3440d1") : references non-existent secret key: ca.crt
Sep 15 16:34:14.492339 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:14.492270    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/80dc5400-d63b-4309-82cb-655318fda275-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-slq96\" (UID: \"80dc5400-d63b-4309-82cb-655318fda275\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-slq96"
Sep 15 16:34:14.500733 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:14.500712    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7v5f\" (UniqueName: \"kubernetes.io/projected/80dc5400-d63b-4309-82cb-655318fda275-kube-api-access-m7v5f\") pod \"keda-metrics-apiserver-77857b7b98-slq96\" (UID: \"80dc5400-d63b-4309-82cb-655318fda275\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-slq96"
Sep 15 16:34:14.772077 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:14.772053    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-rgfmz"]
Sep 15 16:34:14.789900 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:14.789875    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-6c4d864b86-rgfmz"
Sep 15 16:34:14.790145 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:14.789896    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-rgfmz"]
Sep 15 16:34:14.792515 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:14.792497    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Sep 15 16:34:14.895070 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:14.895045    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/579abf00-70f0-467c-970d-639426ae0158-certificates\") pod \"keda-admission-6c4d864b86-rgfmz\" (UID: \"579abf00-70f0-467c-970d-639426ae0158\") " pod="openshift-keda/keda-admission-6c4d864b86-rgfmz"
Sep 15 16:34:14.895157 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:14.895132    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h5bt\" (UniqueName: \"kubernetes.io/projected/579abf00-70f0-467c-970d-639426ae0158-kube-api-access-7h5bt\") pod \"keda-admission-6c4d864b86-rgfmz\" (UID: \"579abf00-70f0-467c-970d-639426ae0158\") " pod="openshift-keda/keda-admission-6c4d864b86-rgfmz"
Sep 15 16:34:14.996069 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:14.996047    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h5bt\" (UniqueName: \"kubernetes.io/projected/579abf00-70f0-467c-970d-639426ae0158-kube-api-access-7h5bt\") pod \"keda-admission-6c4d864b86-rgfmz\" (UID: \"579abf00-70f0-467c-970d-639426ae0158\") " pod="openshift-keda/keda-admission-6c4d864b86-rgfmz"
Sep 15 16:34:14.996157 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:14.996087    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/80dc5400-d63b-4309-82cb-655318fda275-certificates\") pod \"keda-metrics-apiserver-77857b7b98-slq96\" (UID: \"80dc5400-d63b-4309-82cb-655318fda275\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-slq96"
Sep 15 16:34:14.996157 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:14.996131    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/579abf00-70f0-467c-970d-639426ae0158-certificates\") pod \"keda-admission-6c4d864b86-rgfmz\" (UID: \"579abf00-70f0-467c-970d-639426ae0158\") " pod="openshift-keda/keda-admission-6c4d864b86-rgfmz"
Sep 15 16:34:14.996250 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:34:14.996236    2579 projected.go:266] Couldn't get secret openshift-keda/keda-admission-webhooks-certs: secret "keda-admission-webhooks-certs" not found
Sep 15 16:34:14.996283 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:34:14.996259    2579 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-admission-6c4d864b86-rgfmz: secret "keda-admission-webhooks-certs" not found
Sep 15 16:34:14.996322 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:34:14.996310    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/579abf00-70f0-467c-970d-639426ae0158-certificates podName:579abf00-70f0-467c-970d-639426ae0158 nodeName:}" failed. No retries permitted until 2026-09-15 16:34:15.496294084 +0000 UTC m=+472.125948427 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/579abf00-70f0-467c-970d-639426ae0158-certificates") pod "keda-admission-6c4d864b86-rgfmz" (UID: "579abf00-70f0-467c-970d-639426ae0158") : secret "keda-admission-webhooks-certs" not found
Sep 15 16:34:14.998677 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:14.998656    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/80dc5400-d63b-4309-82cb-655318fda275-certificates\") pod \"keda-metrics-apiserver-77857b7b98-slq96\" (UID: \"80dc5400-d63b-4309-82cb-655318fda275\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-slq96"
Sep 15 16:34:15.008352 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:15.008334    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h5bt\" (UniqueName: \"kubernetes.io/projected/579abf00-70f0-467c-970d-639426ae0158-kube-api-access-7h5bt\") pod \"keda-admission-6c4d864b86-rgfmz\" (UID: \"579abf00-70f0-467c-970d-639426ae0158\") " pod="openshift-keda/keda-admission-6c4d864b86-rgfmz"
Sep 15 16:34:15.203600 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:15.203580    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-slq96"
Sep 15 16:34:15.500463 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:15.500402    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/579abf00-70f0-467c-970d-639426ae0158-certificates\") pod \"keda-admission-6c4d864b86-rgfmz\" (UID: \"579abf00-70f0-467c-970d-639426ae0158\") " pod="openshift-keda/keda-admission-6c4d864b86-rgfmz"
Sep 15 16:34:15.500463 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:15.500434    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e725bb1a-676c-4091-9129-924b1d3440d1-certificates\") pod \"keda-operator-579c8958f7-7ns65\" (UID: \"e725bb1a-676c-4091-9129-924b1d3440d1\") " pod="openshift-keda/keda-operator-579c8958f7-7ns65"
Sep 15 16:34:15.502716 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:15.502689    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/579abf00-70f0-467c-970d-639426ae0158-certificates\") pod \"keda-admission-6c4d864b86-rgfmz\" (UID: \"579abf00-70f0-467c-970d-639426ae0158\") " pod="openshift-keda/keda-admission-6c4d864b86-rgfmz"
Sep 15 16:34:15.502863 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:15.502731    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e725bb1a-676c-4091-9129-924b1d3440d1-certificates\") pod \"keda-operator-579c8958f7-7ns65\" (UID: \"e725bb1a-676c-4091-9129-924b1d3440d1\") " pod="openshift-keda/keda-operator-579c8958f7-7ns65"
Sep 15 16:34:15.527622 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:15.527604    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-slq96"]
Sep 15 16:34:15.529817 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:34:15.529777    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80dc5400_d63b_4309_82cb_655318fda275.slice/crio-cd7ebf9e9dc9e9ce82d7927dce0a71eeb0992bcadeb889a1b7f50bab37928549 WatchSource:0}: Error finding container cd7ebf9e9dc9e9ce82d7927dce0a71eeb0992bcadeb889a1b7f50bab37928549: Status 404 returned error can't find the container with id cd7ebf9e9dc9e9ce82d7927dce0a71eeb0992bcadeb889a1b7f50bab37928549
Sep 15 16:34:15.614518 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:15.614501    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-579c8958f7-7ns65"
Sep 15 16:34:15.701064 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:15.701028    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-6c4d864b86-rgfmz"
Sep 15 16:34:15.705174 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:15.705146    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-slq96" event={"ID":"80dc5400-d63b-4309-82cb-655318fda275","Type":"ContainerStarted","Data":"cd7ebf9e9dc9e9ce82d7927dce0a71eeb0992bcadeb889a1b7f50bab37928549"}
Sep 15 16:34:15.912930 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:15.912907    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-579c8958f7-7ns65"]
Sep 15 16:34:15.913517 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:34:15.913495    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode725bb1a_676c_4091_9129_924b1d3440d1.slice/crio-100e354f16f75a73c1731a54a37500fd444765bd11c624facb2965b3fa1c5ca0 WatchSource:0}: Error finding container 100e354f16f75a73c1731a54a37500fd444765bd11c624facb2965b3fa1c5ca0: Status 404 returned error can't find the container with id 100e354f16f75a73c1731a54a37500fd444765bd11c624facb2965b3fa1c5ca0
Sep 15 16:34:15.995138 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:15.995113    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-rgfmz"]
Sep 15 16:34:15.997947 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:34:15.997910    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod579abf00_70f0_467c_970d_639426ae0158.slice/crio-2e1bc52008b49105a73c78dbf7a4971f8f86735f97b15a22141c188624878113 WatchSource:0}: Error finding container 2e1bc52008b49105a73c78dbf7a4971f8f86735f97b15a22141c188624878113: Status 404 returned error can't find the container with id 2e1bc52008b49105a73c78dbf7a4971f8f86735f97b15a22141c188624878113
Sep 15 16:34:16.709805 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:16.709759    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-6c4d864b86-rgfmz" event={"ID":"579abf00-70f0-467c-970d-639426ae0158","Type":"ContainerStarted","Data":"2e1bc52008b49105a73c78dbf7a4971f8f86735f97b15a22141c188624878113"}
Sep 15 16:34:16.710894 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:16.710867    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-579c8958f7-7ns65" event={"ID":"e725bb1a-676c-4091-9129-924b1d3440d1","Type":"ContainerStarted","Data":"100e354f16f75a73c1731a54a37500fd444765bd11c624facb2965b3fa1c5ca0"}
Sep 15 16:34:18.724432 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:18.724398    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-6c4d864b86-rgfmz" event={"ID":"579abf00-70f0-467c-970d-639426ae0158","Type":"ContainerStarted","Data":"07378392ba2dc1cf9639cf57546e36b6fc6863301b95ddf482b85c398ba2c6fe"}
Sep 15 16:34:18.724893 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:18.724510    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-6c4d864b86-rgfmz"
Sep 15 16:34:18.743122 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:18.743070    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-6c4d864b86-rgfmz" podStartSLOduration=2.81084493 podStartE2EDuration="4.743050983s" totalImagesPullingTime="1.932206053s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 16:34:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:34:18.740533527 +0000 UTC m=+475.370187877" watchObservedRunningTime="2026-09-15 16:34:18.743050983 +0000 UTC m=+475.372705331"
Sep 15 16:34:20.737858 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:20.737824    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-579c8958f7-7ns65" event={"ID":"e725bb1a-676c-4091-9129-924b1d3440d1","Type":"ContainerStarted","Data":"3d7416842f126890ddc3d599581fdcc040a8e03ada50e95295ab8fee22215f78"}
Sep 15 16:34:20.738277 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:20.738027    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-579c8958f7-7ns65"
Sep 15 16:34:20.739514 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:20.739489    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-slq96" event={"ID":"80dc5400-d63b-4309-82cb-655318fda275","Type":"ContainerStarted","Data":"8762924da3a863ecc97858be4b70fd99a7887eccfad902fd15393fdaf8102cb9"}
Sep 15 16:34:20.739628 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:20.739550    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-slq96"
Sep 15 16:34:20.755586 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:20.755538    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-579c8958f7-7ns65" podStartSLOduration=3.756094295 podStartE2EDuration="7.755526005s" totalImagesPullingTime="3.99943171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 16:34:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:34:20.753760628 +0000 UTC m=+477.383414980" watchObservedRunningTime="2026-09-15 16:34:20.755526005 +0000 UTC m=+477.385180355"
Sep 15 16:34:20.777392 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:20.777353    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-slq96" podStartSLOduration=2.397074657 podStartE2EDuration="6.777340164s" totalImagesPullingTime="4.380265507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 16:34:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:34:20.776033262 +0000 UTC m=+477.405687613" watchObservedRunningTime="2026-09-15 16:34:20.777340164 +0000 UTC m=+477.406994514"
Sep 15 16:34:31.751664 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:31.751634    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-slq96"
Sep 15 16:34:34.700865 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:34.700834    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-hjdpd"
Sep 15 16:34:39.732277 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:39.732247    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-6c4d864b86-rgfmz"
Sep 15 16:34:41.749859 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:34:41.749833    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-579c8958f7-7ns65"
Sep 15 16:35:31.446592 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:31.446522    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-b589c8cc8-jbj7k"]
Sep 15 16:35:31.448856 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:31.448842    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-b589c8cc8-jbj7k"
Sep 15 16:35:31.458468 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:31.458448    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Sep 15 16:35:31.459421 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:31.459399    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Sep 15 16:35:31.459514 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:31.459425    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-wln7n\""
Sep 15 16:35:31.459514 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:31.459427    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Sep 15 16:35:31.481677 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:31.481657    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-645f9c8fd6-gfxnj"]
Sep 15 16:35:31.483844 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:31.483825    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-645f9c8fd6-gfxnj"
Sep 15 16:35:31.486280 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:31.486249    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-b589c8cc8-jbj7k"]
Sep 15 16:35:31.488261 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:31.488243    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Sep 15 16:35:31.488447 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:31.488424    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-s9454\""
Sep 15 16:35:31.511963 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:31.511942    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-645f9c8fd6-gfxnj"]
Sep 15 16:35:31.635118 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:31.635094    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vk6fn\" (UniqueName: \"kubernetes.io/projected/8a6e8cf1-3572-4a66-a933-2913ff876170-kube-api-access-vk6fn\") pod \"kserve-controller-manager-b589c8cc8-jbj7k\" (UID: \"8a6e8cf1-3572-4a66-a933-2913ff876170\") " pod="kserve/kserve-controller-manager-b589c8cc8-jbj7k"
Sep 15 16:35:31.635202 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:31.635148    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/05583e14-ad32-4f6a-a34c-4b1d819da845-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-gfxnj\" (UID: \"05583e14-ad32-4f6a-a34c-4b1d819da845\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-gfxnj"
Sep 15 16:35:31.635202 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:31.635174    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75xhd\" (UniqueName: \"kubernetes.io/projected/05583e14-ad32-4f6a-a34c-4b1d819da845-kube-api-access-75xhd\") pod \"llmisvc-controller-manager-645f9c8fd6-gfxnj\" (UID: \"05583e14-ad32-4f6a-a34c-4b1d819da845\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-gfxnj"
Sep 15 16:35:31.635269 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:31.635210    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a6e8cf1-3572-4a66-a933-2913ff876170-cert\") pod \"kserve-controller-manager-b589c8cc8-jbj7k\" (UID: \"8a6e8cf1-3572-4a66-a933-2913ff876170\") " pod="kserve/kserve-controller-manager-b589c8cc8-jbj7k"
Sep 15 16:35:31.735538 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:31.735515    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vk6fn\" (UniqueName: \"kubernetes.io/projected/8a6e8cf1-3572-4a66-a933-2913ff876170-kube-api-access-vk6fn\") pod \"kserve-controller-manager-b589c8cc8-jbj7k\" (UID: \"8a6e8cf1-3572-4a66-a933-2913ff876170\") " pod="kserve/kserve-controller-manager-b589c8cc8-jbj7k"
Sep 15 16:35:31.735633 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:31.735578    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/05583e14-ad32-4f6a-a34c-4b1d819da845-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-gfxnj\" (UID: \"05583e14-ad32-4f6a-a34c-4b1d819da845\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-gfxnj"
Sep 15 16:35:31.735633 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:31.735603    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75xhd\" (UniqueName: \"kubernetes.io/projected/05583e14-ad32-4f6a-a34c-4b1d819da845-kube-api-access-75xhd\") pod \"llmisvc-controller-manager-645f9c8fd6-gfxnj\" (UID: \"05583e14-ad32-4f6a-a34c-4b1d819da845\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-gfxnj"
Sep 15 16:35:31.735730 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:31.735639    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a6e8cf1-3572-4a66-a933-2913ff876170-cert\") pod \"kserve-controller-manager-b589c8cc8-jbj7k\" (UID: \"8a6e8cf1-3572-4a66-a933-2913ff876170\") " pod="kserve/kserve-controller-manager-b589c8cc8-jbj7k"
Sep 15 16:35:31.735799 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:35:31.735742    2579 secret.go:189] Couldn't get secret kserve/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 15 16:35:31.735855 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:35:31.735824    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a6e8cf1-3572-4a66-a933-2913ff876170-cert podName:8a6e8cf1-3572-4a66-a933-2913ff876170 nodeName:}" failed. No retries permitted until 2026-09-15 16:35:32.235805795 +0000 UTC m=+548.865460141 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8a6e8cf1-3572-4a66-a933-2913ff876170-cert") pod "kserve-controller-manager-b589c8cc8-jbj7k" (UID: "8a6e8cf1-3572-4a66-a933-2913ff876170") : secret "kserve-webhook-server-cert" not found
Sep 15 16:35:31.738015 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:31.737992    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/05583e14-ad32-4f6a-a34c-4b1d819da845-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-gfxnj\" (UID: \"05583e14-ad32-4f6a-a34c-4b1d819da845\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-gfxnj"
Sep 15 16:35:31.745697 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:31.745667    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vk6fn\" (UniqueName: \"kubernetes.io/projected/8a6e8cf1-3572-4a66-a933-2913ff876170-kube-api-access-vk6fn\") pod \"kserve-controller-manager-b589c8cc8-jbj7k\" (UID: \"8a6e8cf1-3572-4a66-a933-2913ff876170\") " pod="kserve/kserve-controller-manager-b589c8cc8-jbj7k"
Sep 15 16:35:31.746658 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:31.746638    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75xhd\" (UniqueName: \"kubernetes.io/projected/05583e14-ad32-4f6a-a34c-4b1d819da845-kube-api-access-75xhd\") pod \"llmisvc-controller-manager-645f9c8fd6-gfxnj\" (UID: \"05583e14-ad32-4f6a-a34c-4b1d819da845\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-gfxnj"
Sep 15 16:35:31.795799 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:31.795601    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-645f9c8fd6-gfxnj"
Sep 15 16:35:32.067892 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:32.067870    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-645f9c8fd6-gfxnj"]
Sep 15 16:35:32.070140 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:35:32.070121    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod05583e14_ad32_4f6a_a34c_4b1d819da845.slice/crio-3528d2d22a19ec1d383f731a9accf350ec40771d3765aba9fbdb04acbb622d3a WatchSource:0}: Error finding container 3528d2d22a19ec1d383f731a9accf350ec40771d3765aba9fbdb04acbb622d3a: Status 404 returned error can't find the container with id 3528d2d22a19ec1d383f731a9accf350ec40771d3765aba9fbdb04acbb622d3a
Sep 15 16:35:32.238982 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:32.238959    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a6e8cf1-3572-4a66-a933-2913ff876170-cert\") pod \"kserve-controller-manager-b589c8cc8-jbj7k\" (UID: \"8a6e8cf1-3572-4a66-a933-2913ff876170\") " pod="kserve/kserve-controller-manager-b589c8cc8-jbj7k"
Sep 15 16:35:32.241645 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:32.241630    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a6e8cf1-3572-4a66-a933-2913ff876170-cert\") pod \"kserve-controller-manager-b589c8cc8-jbj7k\" (UID: \"8a6e8cf1-3572-4a66-a933-2913ff876170\") " pod="kserve/kserve-controller-manager-b589c8cc8-jbj7k"
Sep 15 16:35:32.358315 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:32.358275    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-b589c8cc8-jbj7k"
Sep 15 16:35:32.624202 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:32.624140    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-b589c8cc8-jbj7k"]
Sep 15 16:35:32.626701 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:35:32.626677    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a6e8cf1_3572_4a66_a933_2913ff876170.slice/crio-2b92b9c187ecf14e0c0661911152b73d3dcf967f94667fbfdf036612d3b3ff65 WatchSource:0}: Error finding container 2b92b9c187ecf14e0c0661911152b73d3dcf967f94667fbfdf036612d3b3ff65: Status 404 returned error can't find the container with id 2b92b9c187ecf14e0c0661911152b73d3dcf967f94667fbfdf036612d3b3ff65
Sep 15 16:35:33.028143 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:33.028116    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-645f9c8fd6-gfxnj" event={"ID":"05583e14-ad32-4f6a-a34c-4b1d819da845","Type":"ContainerStarted","Data":"3528d2d22a19ec1d383f731a9accf350ec40771d3765aba9fbdb04acbb622d3a"}
Sep 15 16:35:33.029191 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:33.029170    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-b589c8cc8-jbj7k" event={"ID":"8a6e8cf1-3572-4a66-a933-2913ff876170","Type":"ContainerStarted","Data":"2b92b9c187ecf14e0c0661911152b73d3dcf967f94667fbfdf036612d3b3ff65"}
Sep 15 16:35:37.050059 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:37.049980    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-645f9c8fd6-gfxnj" event={"ID":"05583e14-ad32-4f6a-a34c-4b1d819da845","Type":"ContainerStarted","Data":"72b0e242d12824f61cd420ba4915510a1df517631451aa5b3c8e2a503e5ebc46"}
Sep 15 16:35:37.050431 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:37.050104    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-645f9c8fd6-gfxnj"
Sep 15 16:35:37.051490 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:37.051462    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-b589c8cc8-jbj7k" event={"ID":"8a6e8cf1-3572-4a66-a933-2913ff876170","Type":"ContainerStarted","Data":"140dfc764e9eacdded52bdbf2b547c9ef68949526b02cc5906b44ec26d3fed23"}
Sep 15 16:35:37.051596 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:37.051583    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-b589c8cc8-jbj7k"
Sep 15 16:35:37.066037 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:37.065998    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-645f9c8fd6-gfxnj" podStartSLOduration=1.3748649290000001 podStartE2EDuration="6.065984815s" totalImagesPullingTime="4.691119886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 16:35:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:35:37.064871866 +0000 UTC m=+553.694526214" watchObservedRunningTime="2026-09-15 16:35:37.065984815 +0000 UTC m=+553.695639160"
Sep 15 16:35:37.079373 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:35:37.079338    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-b589c8cc8-jbj7k" podStartSLOduration=1.988133247 podStartE2EDuration="6.079327282s" totalImagesPullingTime="4.091194035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 16:35:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:35:37.078860246 +0000 UTC m=+553.708514596" watchObservedRunningTime="2026-09-15 16:35:37.079327282 +0000 UTC m=+553.708981632"
Sep 15 16:36:08.057735 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:08.057704    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-645f9c8fd6-gfxnj"
Sep 15 16:36:08.060749 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:08.060730    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-b589c8cc8-jbj7k"
Sep 15 16:36:15.948147 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:15.948115    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-b589c8cc8-jbj7k"]
Sep 15 16:36:15.948547 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:15.948301    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve/kserve-controller-manager-b589c8cc8-jbj7k" podUID="8a6e8cf1-3572-4a66-a933-2913ff876170" containerName="manager" containerID="cri-o://140dfc764e9eacdded52bdbf2b547c9ef68949526b02cc5906b44ec26d3fed23" gracePeriod=10
Sep 15 16:36:15.970143 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:15.970120    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-b589c8cc8-gntnd"]
Sep 15 16:36:15.972191 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:15.972178    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-b589c8cc8-gntnd"
Sep 15 16:36:15.983397 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:15.983377    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-b589c8cc8-gntnd"]
Sep 15 16:36:16.050501 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:16.050478    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c61a1eff-9292-4700-a2c7-deb93bf22e86-cert\") pod \"kserve-controller-manager-b589c8cc8-gntnd\" (UID: \"c61a1eff-9292-4700-a2c7-deb93bf22e86\") " pod="kserve/kserve-controller-manager-b589c8cc8-gntnd"
Sep 15 16:36:16.050585 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:16.050560    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtg2t\" (UniqueName: \"kubernetes.io/projected/c61a1eff-9292-4700-a2c7-deb93bf22e86-kube-api-access-rtg2t\") pod \"kserve-controller-manager-b589c8cc8-gntnd\" (UID: \"c61a1eff-9292-4700-a2c7-deb93bf22e86\") " pod="kserve/kserve-controller-manager-b589c8cc8-gntnd"
Sep 15 16:36:16.151463 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:16.151435    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtg2t\" (UniqueName: \"kubernetes.io/projected/c61a1eff-9292-4700-a2c7-deb93bf22e86-kube-api-access-rtg2t\") pod \"kserve-controller-manager-b589c8cc8-gntnd\" (UID: \"c61a1eff-9292-4700-a2c7-deb93bf22e86\") " pod="kserve/kserve-controller-manager-b589c8cc8-gntnd"
Sep 15 16:36:16.151600 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:16.151482    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c61a1eff-9292-4700-a2c7-deb93bf22e86-cert\") pod \"kserve-controller-manager-b589c8cc8-gntnd\" (UID: \"c61a1eff-9292-4700-a2c7-deb93bf22e86\") " pod="kserve/kserve-controller-manager-b589c8cc8-gntnd"
Sep 15 16:36:16.153958 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:16.153935    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c61a1eff-9292-4700-a2c7-deb93bf22e86-cert\") pod \"kserve-controller-manager-b589c8cc8-gntnd\" (UID: \"c61a1eff-9292-4700-a2c7-deb93bf22e86\") " pod="kserve/kserve-controller-manager-b589c8cc8-gntnd"
Sep 15 16:36:16.160215 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:16.160191    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtg2t\" (UniqueName: \"kubernetes.io/projected/c61a1eff-9292-4700-a2c7-deb93bf22e86-kube-api-access-rtg2t\") pod \"kserve-controller-manager-b589c8cc8-gntnd\" (UID: \"c61a1eff-9292-4700-a2c7-deb93bf22e86\") " pod="kserve/kserve-controller-manager-b589c8cc8-gntnd"
Sep 15 16:36:16.178476 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:16.178457    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-b589c8cc8-jbj7k"
Sep 15 16:36:16.205040 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:16.205017    2579 generic.go:356] "Generic (PLEG): container finished" podID="8a6e8cf1-3572-4a66-a933-2913ff876170" containerID="140dfc764e9eacdded52bdbf2b547c9ef68949526b02cc5906b44ec26d3fed23" exitCode=0
Sep 15 16:36:16.205157 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:16.205072    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-b589c8cc8-jbj7k"
Sep 15 16:36:16.205157 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:16.205098    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-b589c8cc8-jbj7k" event={"ID":"8a6e8cf1-3572-4a66-a933-2913ff876170","Type":"ContainerDied","Data":"140dfc764e9eacdded52bdbf2b547c9ef68949526b02cc5906b44ec26d3fed23"}
Sep 15 16:36:16.205236 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:16.205158    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-b589c8cc8-jbj7k" event={"ID":"8a6e8cf1-3572-4a66-a933-2913ff876170","Type":"ContainerDied","Data":"2b92b9c187ecf14e0c0661911152b73d3dcf967f94667fbfdf036612d3b3ff65"}
Sep 15 16:36:16.205236 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:16.205187    2579 scope.go:117] "RemoveContainer" containerID="140dfc764e9eacdded52bdbf2b547c9ef68949526b02cc5906b44ec26d3fed23"
Sep 15 16:36:16.213989 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:16.213956    2579 scope.go:117] "RemoveContainer" containerID="140dfc764e9eacdded52bdbf2b547c9ef68949526b02cc5906b44ec26d3fed23"
Sep 15 16:36:16.214315 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:36:16.214291    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"140dfc764e9eacdded52bdbf2b547c9ef68949526b02cc5906b44ec26d3fed23\": container with ID starting with 140dfc764e9eacdded52bdbf2b547c9ef68949526b02cc5906b44ec26d3fed23 not found: ID does not exist" containerID="140dfc764e9eacdded52bdbf2b547c9ef68949526b02cc5906b44ec26d3fed23"
Sep 15 16:36:16.214393 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:16.214326    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"140dfc764e9eacdded52bdbf2b547c9ef68949526b02cc5906b44ec26d3fed23"} err="failed to get container status \"140dfc764e9eacdded52bdbf2b547c9ef68949526b02cc5906b44ec26d3fed23\": rpc error: code = NotFound desc = could not find container \"140dfc764e9eacdded52bdbf2b547c9ef68949526b02cc5906b44ec26d3fed23\": container with ID starting with 140dfc764e9eacdded52bdbf2b547c9ef68949526b02cc5906b44ec26d3fed23 not found: ID does not exist"
Sep 15 16:36:16.251747 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:16.251721    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vk6fn\" (UniqueName: \"kubernetes.io/projected/8a6e8cf1-3572-4a66-a933-2913ff876170-kube-api-access-vk6fn\") pod \"8a6e8cf1-3572-4a66-a933-2913ff876170\" (UID: \"8a6e8cf1-3572-4a66-a933-2913ff876170\") "
Sep 15 16:36:16.251889 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:16.251757    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a6e8cf1-3572-4a66-a933-2913ff876170-cert\") pod \"8a6e8cf1-3572-4a66-a933-2913ff876170\" (UID: \"8a6e8cf1-3572-4a66-a933-2913ff876170\") "
Sep 15 16:36:16.254031 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:16.254007    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a6e8cf1-3572-4a66-a933-2913ff876170-cert" (OuterVolumeSpecName: "cert") pod "8a6e8cf1-3572-4a66-a933-2913ff876170" (UID: "8a6e8cf1-3572-4a66-a933-2913ff876170"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 16:36:16.254136 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:16.254076    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a6e8cf1-3572-4a66-a933-2913ff876170-kube-api-access-vk6fn" (OuterVolumeSpecName: "kube-api-access-vk6fn") pod "8a6e8cf1-3572-4a66-a933-2913ff876170" (UID: "8a6e8cf1-3572-4a66-a933-2913ff876170"). InnerVolumeSpecName "kube-api-access-vk6fn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 16:36:16.326122 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:16.326089    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-b589c8cc8-gntnd"
Sep 15 16:36:16.352432 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:16.352397    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vk6fn\" (UniqueName: \"kubernetes.io/projected/8a6e8cf1-3572-4a66-a933-2913ff876170-kube-api-access-vk6fn\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:36:16.352432 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:16.352436    2579 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a6e8cf1-3572-4a66-a933-2913ff876170-cert\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:36:16.525917 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:16.525889    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-b589c8cc8-jbj7k"]
Sep 15 16:36:16.530003 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:16.529979    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-b589c8cc8-jbj7k"]
Sep 15 16:36:16.596973 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:16.596944    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-b589c8cc8-gntnd"]
Sep 15 16:36:16.599714 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:36:16.599689    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc61a1eff_9292_4700_a2c7_deb93bf22e86.slice/crio-109e19c83693eb9c8442961ba140733e94b0d59a62019331532baf73b39b1afb WatchSource:0}: Error finding container 109e19c83693eb9c8442961ba140733e94b0d59a62019331532baf73b39b1afb: Status 404 returned error can't find the container with id 109e19c83693eb9c8442961ba140733e94b0d59a62019331532baf73b39b1afb
Sep 15 16:36:17.211762 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:17.211729    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-b589c8cc8-gntnd" event={"ID":"c61a1eff-9292-4700-a2c7-deb93bf22e86","Type":"ContainerStarted","Data":"ec0ed12a755880fb43f6395c26482c8c5b4850d535d3911fd779e30a7b8afe5b"}
Sep 15 16:36:17.212217 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:17.211905    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-b589c8cc8-gntnd"
Sep 15 16:36:17.212217 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:17.211963    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-b589c8cc8-gntnd" event={"ID":"c61a1eff-9292-4700-a2c7-deb93bf22e86","Type":"ContainerStarted","Data":"109e19c83693eb9c8442961ba140733e94b0d59a62019331532baf73b39b1afb"}
Sep 15 16:36:17.227834 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:17.227771    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-b589c8cc8-gntnd" podStartSLOduration=1.9416023610000002 podStartE2EDuration="2.227756048s" totalImagesPullingTime="286.153687ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 16:36:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:36:17.226918118 +0000 UTC m=+593.856572470" watchObservedRunningTime="2026-09-15 16:36:17.227756048 +0000 UTC m=+593.857410399"
Sep 15 16:36:17.909048 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:17.909017    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a6e8cf1-3572-4a66-a933-2913ff876170" path="/var/lib/kubelet/pods/8a6e8cf1-3572-4a66-a933-2913ff876170/volumes"
Sep 15 16:36:23.831341 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:23.831314    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4rdn_44ca88f0-c149-443e-a2d1-a30964775818/ovn-acl-logging/0.log"
Sep 15 16:36:23.832451 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:23.832435    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4rdn_44ca88f0-c149-443e-a2d1-a30964775818/ovn-acl-logging/0.log"
Sep 15 16:36:48.221981 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:36:48.221904    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-b589c8cc8-gntnd"
Sep 15 16:38:38.479151 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:38.479077    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d"]
Sep 15 16:38:38.479677 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:38.479430    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a6e8cf1-3572-4a66-a933-2913ff876170" containerName="manager"
Sep 15 16:38:38.479677 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:38.479445    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a6e8cf1-3572-4a66-a933-2913ff876170" containerName="manager"
Sep 15 16:38:38.479677 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:38.479497    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a6e8cf1-3572-4a66-a933-2913ff876170" containerName="manager"
Sep 15 16:38:38.482849 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:38.482827    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d"
Sep 15 16:38:38.485930 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:38.485909    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-rollback-kube-rbac-proxy-sar-config\""
Sep 15 16:38:38.486052 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:38.485951    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Sep 15 16:38:38.486218 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:38.486203    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 15 16:38:38.486533 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:38.486519    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 15 16:38:38.486623 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:38.486532    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-lqw4s\""
Sep 15 16:38:38.487000 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:38.486978    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-rollback-predictor-serving-cert\""
Sep 15 16:38:38.487000 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:38.486996    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Sep 15 16:38:38.496350 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:38.496325    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d"]
Sep 15 16:38:38.599643 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:38.599622    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbxg6\" (UniqueName: \"kubernetes.io/projected/71830964-3306-41dd-98d9-861abf40b42b-kube-api-access-kbxg6\") pod \"isvc-canary-rollback-predictor-6b76f97c96-kkd7d\" (UID: \"71830964-3306-41dd-98d9-861abf40b42b\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d"
Sep 15 16:38:38.599772 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:38.599675    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/71830964-3306-41dd-98d9-861abf40b42b-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-6b76f97c96-kkd7d\" (UID: \"71830964-3306-41dd-98d9-861abf40b42b\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d"
Sep 15 16:38:38.599772 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:38.599705    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/71830964-3306-41dd-98d9-861abf40b42b-proxy-tls\") pod \"isvc-canary-rollback-predictor-6b76f97c96-kkd7d\" (UID: \"71830964-3306-41dd-98d9-861abf40b42b\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d"
Sep 15 16:38:38.599772 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:38.599731    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/71830964-3306-41dd-98d9-861abf40b42b-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-6b76f97c96-kkd7d\" (UID: \"71830964-3306-41dd-98d9-861abf40b42b\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d"
Sep 15 16:38:38.599937 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:38.599803    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/71830964-3306-41dd-98d9-861abf40b42b-cabundle-cert\") pod \"isvc-canary-rollback-predictor-6b76f97c96-kkd7d\" (UID: \"71830964-3306-41dd-98d9-861abf40b42b\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d"
Sep 15 16:38:38.700692 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:38.700670    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbxg6\" (UniqueName: \"kubernetes.io/projected/71830964-3306-41dd-98d9-861abf40b42b-kube-api-access-kbxg6\") pod \"isvc-canary-rollback-predictor-6b76f97c96-kkd7d\" (UID: \"71830964-3306-41dd-98d9-861abf40b42b\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d"
Sep 15 16:38:38.700816 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:38.700704    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/71830964-3306-41dd-98d9-861abf40b42b-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-6b76f97c96-kkd7d\" (UID: \"71830964-3306-41dd-98d9-861abf40b42b\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d"
Sep 15 16:38:38.700816 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:38.700726    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/71830964-3306-41dd-98d9-861abf40b42b-proxy-tls\") pod \"isvc-canary-rollback-predictor-6b76f97c96-kkd7d\" (UID: \"71830964-3306-41dd-98d9-861abf40b42b\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d"
Sep 15 16:38:38.700816 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:38.700741    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/71830964-3306-41dd-98d9-861abf40b42b-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-6b76f97c96-kkd7d\" (UID: \"71830964-3306-41dd-98d9-861abf40b42b\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d"
Sep 15 16:38:38.700970 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:38.700881    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/71830964-3306-41dd-98d9-861abf40b42b-cabundle-cert\") pod \"isvc-canary-rollback-predictor-6b76f97c96-kkd7d\" (UID: \"71830964-3306-41dd-98d9-861abf40b42b\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d"
Sep 15 16:38:38.701158 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:38.701136    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/71830964-3306-41dd-98d9-861abf40b42b-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-6b76f97c96-kkd7d\" (UID: \"71830964-3306-41dd-98d9-861abf40b42b\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d"
Sep 15 16:38:38.701398 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:38.701375    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/71830964-3306-41dd-98d9-861abf40b42b-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-6b76f97c96-kkd7d\" (UID: \"71830964-3306-41dd-98d9-861abf40b42b\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d"
Sep 15 16:38:38.701463 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:38.701432    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/71830964-3306-41dd-98d9-861abf40b42b-cabundle-cert\") pod \"isvc-canary-rollback-predictor-6b76f97c96-kkd7d\" (UID: \"71830964-3306-41dd-98d9-861abf40b42b\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d"
Sep 15 16:38:38.703385 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:38.703371    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/71830964-3306-41dd-98d9-861abf40b42b-proxy-tls\") pod \"isvc-canary-rollback-predictor-6b76f97c96-kkd7d\" (UID: \"71830964-3306-41dd-98d9-861abf40b42b\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d"
Sep 15 16:38:38.709078 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:38.709057    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbxg6\" (UniqueName: \"kubernetes.io/projected/71830964-3306-41dd-98d9-861abf40b42b-kube-api-access-kbxg6\") pod \"isvc-canary-rollback-predictor-6b76f97c96-kkd7d\" (UID: \"71830964-3306-41dd-98d9-861abf40b42b\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d"
Sep 15 16:38:38.793966 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:38.793907    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d"
Sep 15 16:38:39.066335 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:39.066310    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d"]
Sep 15 16:38:39.068926 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:38:39.068903    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71830964_3306_41dd_98d9_861abf40b42b.slice/crio-742480abbd7f6df2387da593270f6b4d55ab2800fca9b8c031d3e960a7104f60 WatchSource:0}: Error finding container 742480abbd7f6df2387da593270f6b4d55ab2800fca9b8c031d3e960a7104f60: Status 404 returned error can't find the container with id 742480abbd7f6df2387da593270f6b4d55ab2800fca9b8c031d3e960a7104f60
Sep 15 16:38:39.799327 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:39.799293    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d" event={"ID":"71830964-3306-41dd-98d9-861abf40b42b","Type":"ContainerStarted","Data":"742480abbd7f6df2387da593270f6b4d55ab2800fca9b8c031d3e960a7104f60"}
Sep 15 16:38:42.816279 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:42.816239    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d" event={"ID":"71830964-3306-41dd-98d9-861abf40b42b","Type":"ContainerStarted","Data":"69c901cc003c67b1b98f7df51628d650b9b633386070496c5829138f74ab6d02"}
Sep 15 16:38:46.833035 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:46.833006    2579 generic.go:356] "Generic (PLEG): container finished" podID="71830964-3306-41dd-98d9-861abf40b42b" containerID="69c901cc003c67b1b98f7df51628d650b9b633386070496c5829138f74ab6d02" exitCode=0
Sep 15 16:38:46.833323 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:46.833060    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d" event={"ID":"71830964-3306-41dd-98d9-861abf40b42b","Type":"ContainerDied","Data":"69c901cc003c67b1b98f7df51628d650b9b633386070496c5829138f74ab6d02"}
Sep 15 16:38:46.834021 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:46.833998    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 16:38:59.910864 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:38:59.910819    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d" event={"ID":"71830964-3306-41dd-98d9-861abf40b42b","Type":"ContainerStarted","Data":"9211f278b506c71f76ec4151937022716020cdf0680f7e18b57338fa9ae1caec"}
Sep 15 16:39:01.919475 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:39:01.919444    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d" event={"ID":"71830964-3306-41dd-98d9-861abf40b42b","Type":"ContainerStarted","Data":"ebd96f52d195e0f0b85f6b10133547fc3e51695d9747fe6d2034569153472c41"}
Sep 15 16:39:01.919893 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:39:01.919575    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d"
Sep 15 16:39:01.938150 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:39:01.938105    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d" podStartSLOduration=2.5710367659999998 podStartE2EDuration="23.93809212s" totalImagesPullingTime="17.66958606s" totalInitContainerRuntime="3.697469294s" isStatefulPod=false podCreationTimestamp="2026-09-15 16:38:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:39:01.936452101 +0000 UTC m=+758.566106464" watchObservedRunningTime="2026-09-15 16:39:01.93809212 +0000 UTC m=+758.567746470"
Sep 15 16:39:02.923278 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:39:02.923254    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d"
Sep 15 16:39:02.924274 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:39:02.924245    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d" podUID="71830964-3306-41dd-98d9-861abf40b42b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Sep 15 16:39:03.926986 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:39:03.926945    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d" podUID="71830964-3306-41dd-98d9-861abf40b42b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Sep 15 16:39:08.930718 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:39:08.930682    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d"
Sep 15 16:39:08.931207 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:39:08.931180    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d" podUID="71830964-3306-41dd-98d9-861abf40b42b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Sep 15 16:39:18.931335 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:39:18.931293    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d" podUID="71830964-3306-41dd-98d9-861abf40b42b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Sep 15 16:39:28.931598 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:39:28.931526    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d" podUID="71830964-3306-41dd-98d9-861abf40b42b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Sep 15 16:39:38.931273 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:39:38.931237    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d" podUID="71830964-3306-41dd-98d9-861abf40b42b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Sep 15 16:39:48.931528 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:39:48.931449    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d" podUID="71830964-3306-41dd-98d9-861abf40b42b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Sep 15 16:39:58.932068 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:39:58.932021    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d" podUID="71830964-3306-41dd-98d9-861abf40b42b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Sep 15 16:40:08.931932 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:08.931906    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d"
Sep 15 16:40:33.297577 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:33.297548    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7"]
Sep 15 16:40:33.301090 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:33.301069    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7"
Sep 15 16:40:33.303347 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:33.303315    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-raw-e2542-kube-rbac-proxy-sar-config\""
Sep 15 16:40:33.303483 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:33.303466    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-raw-e2542-predictor-serving-cert\""
Sep 15 16:40:33.308120 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:33.308095    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1b077164-aeea-45fb-ae73-dafe18b80859-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7\" (UID: \"1b077164-aeea-45fb-ae73-dafe18b80859\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7"
Sep 15 16:40:33.308203 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:33.308140    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1b077164-aeea-45fb-ae73-dafe18b80859-cabundle-cert\") pod \"isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7\" (UID: \"1b077164-aeea-45fb-ae73-dafe18b80859\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7"
Sep 15 16:40:33.308266 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:33.308209    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2q25\" (UniqueName: \"kubernetes.io/projected/1b077164-aeea-45fb-ae73-dafe18b80859-kube-api-access-t2q25\") pod \"isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7\" (UID: \"1b077164-aeea-45fb-ae73-dafe18b80859\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7"
Sep 15 16:40:33.308266 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:33.308244    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b077164-aeea-45fb-ae73-dafe18b80859-proxy-tls\") pod \"isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7\" (UID: \"1b077164-aeea-45fb-ae73-dafe18b80859\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7"
Sep 15 16:40:33.308365 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:33.308276    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1b077164-aeea-45fb-ae73-dafe18b80859-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7\" (UID: \"1b077164-aeea-45fb-ae73-dafe18b80859\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7"
Sep 15 16:40:33.309772 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:33.309754    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7"]
Sep 15 16:40:33.409180 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:33.409148    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1b077164-aeea-45fb-ae73-dafe18b80859-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7\" (UID: \"1b077164-aeea-45fb-ae73-dafe18b80859\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7"
Sep 15 16:40:33.409307 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:33.409204    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1b077164-aeea-45fb-ae73-dafe18b80859-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7\" (UID: \"1b077164-aeea-45fb-ae73-dafe18b80859\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7"
Sep 15 16:40:33.409307 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:33.409236    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1b077164-aeea-45fb-ae73-dafe18b80859-cabundle-cert\") pod \"isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7\" (UID: \"1b077164-aeea-45fb-ae73-dafe18b80859\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7"
Sep 15 16:40:33.409307 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:33.409301    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2q25\" (UniqueName: \"kubernetes.io/projected/1b077164-aeea-45fb-ae73-dafe18b80859-kube-api-access-t2q25\") pod \"isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7\" (UID: \"1b077164-aeea-45fb-ae73-dafe18b80859\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7"
Sep 15 16:40:33.409448 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:33.409339    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b077164-aeea-45fb-ae73-dafe18b80859-proxy-tls\") pod \"isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7\" (UID: \"1b077164-aeea-45fb-ae73-dafe18b80859\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7"
Sep 15 16:40:33.409617 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:33.409595    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1b077164-aeea-45fb-ae73-dafe18b80859-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7\" (UID: \"1b077164-aeea-45fb-ae73-dafe18b80859\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7"
Sep 15 16:40:33.409910 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:33.409887    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1b077164-aeea-45fb-ae73-dafe18b80859-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7\" (UID: \"1b077164-aeea-45fb-ae73-dafe18b80859\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7"
Sep 15 16:40:33.409987 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:33.409952    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1b077164-aeea-45fb-ae73-dafe18b80859-cabundle-cert\") pod \"isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7\" (UID: \"1b077164-aeea-45fb-ae73-dafe18b80859\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7"
Sep 15 16:40:33.412016 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:33.411996    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b077164-aeea-45fb-ae73-dafe18b80859-proxy-tls\") pod \"isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7\" (UID: \"1b077164-aeea-45fb-ae73-dafe18b80859\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7"
Sep 15 16:40:33.418512 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:33.418492    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2q25\" (UniqueName: \"kubernetes.io/projected/1b077164-aeea-45fb-ae73-dafe18b80859-kube-api-access-t2q25\") pod \"isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7\" (UID: \"1b077164-aeea-45fb-ae73-dafe18b80859\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7"
Sep 15 16:40:33.611767 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:33.611713    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7"
Sep 15 16:40:33.891090 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:33.891069    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7"]
Sep 15 16:40:33.893169 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:40:33.893140    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b077164_aeea_45fb_ae73_dafe18b80859.slice/crio-765c42bebd54b1ad943ddd337896377d4b068b276574bba1303347e3a5d43f3a WatchSource:0}: Error finding container 765c42bebd54b1ad943ddd337896377d4b068b276574bba1303347e3a5d43f3a: Status 404 returned error can't find the container with id 765c42bebd54b1ad943ddd337896377d4b068b276574bba1303347e3a5d43f3a
Sep 15 16:40:34.294480 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:34.294447    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7" event={"ID":"1b077164-aeea-45fb-ae73-dafe18b80859","Type":"ContainerStarted","Data":"77768bd3a6fc74fbfea86d1603c7583bf0b3ca4b2eb6fffa1a0a460fe68d2652"}
Sep 15 16:40:34.294651 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:34.294507    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7" event={"ID":"1b077164-aeea-45fb-ae73-dafe18b80859","Type":"ContainerStarted","Data":"765c42bebd54b1ad943ddd337896377d4b068b276574bba1303347e3a5d43f3a"}
Sep 15 16:40:38.312500 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:38.312469    2579 generic.go:356] "Generic (PLEG): container finished" podID="1b077164-aeea-45fb-ae73-dafe18b80859" containerID="77768bd3a6fc74fbfea86d1603c7583bf0b3ca4b2eb6fffa1a0a460fe68d2652" exitCode=0
Sep 15 16:40:38.312812 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:38.312540    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7" event={"ID":"1b077164-aeea-45fb-ae73-dafe18b80859","Type":"ContainerDied","Data":"77768bd3a6fc74fbfea86d1603c7583bf0b3ca4b2eb6fffa1a0a460fe68d2652"}
Sep 15 16:40:39.319069 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:39.319039    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7" event={"ID":"1b077164-aeea-45fb-ae73-dafe18b80859","Type":"ContainerStarted","Data":"d08508d5d5be9923ea990b75305b4f753f84dc228366141c12a8b2d60bb6023d"}
Sep 15 16:40:39.319466 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:39.319091    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7" event={"ID":"1b077164-aeea-45fb-ae73-dafe18b80859","Type":"ContainerStarted","Data":"5873363d901ef97b51fdcc801ff5feeecbf06c6b353bba210059da921cab572c"}
Sep 15 16:40:39.319466 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:39.319380    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7"
Sep 15 16:40:39.335935 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:39.335889    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7" podStartSLOduration=2.462156034 podStartE2EDuration="6.335875628s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.873719594s" isStatefulPod=false podCreationTimestamp="2026-09-15 16:40:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:40:39.335382911 +0000 UTC m=+855.965037262" watchObservedRunningTime="2026-09-15 16:40:39.335875628 +0000 UTC m=+855.965529978"
Sep 15 16:40:40.323355 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:40.323328    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7"
Sep 15 16:40:40.324466 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:40.324436    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7" podUID="1b077164-aeea-45fb-ae73-dafe18b80859" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Sep 15 16:40:41.327511 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:41.327486    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7" podUID="1b077164-aeea-45fb-ae73-dafe18b80859" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Sep 15 16:40:46.331901 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:46.331877    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7"
Sep 15 16:40:46.332270 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:46.332246    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7" podUID="1b077164-aeea-45fb-ae73-dafe18b80859" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Sep 15 16:40:56.332561 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:40:56.332529    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7" podUID="1b077164-aeea-45fb-ae73-dafe18b80859" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Sep 15 16:41:06.332968 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:41:06.332932    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7" podUID="1b077164-aeea-45fb-ae73-dafe18b80859" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Sep 15 16:41:16.333029 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:41:16.332953    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7" podUID="1b077164-aeea-45fb-ae73-dafe18b80859" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Sep 15 16:41:23.862947 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:41:23.862917    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4rdn_44ca88f0-c149-443e-a2d1-a30964775818/ovn-acl-logging/0.log"
Sep 15 16:41:23.863993 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:41:23.863975    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4rdn_44ca88f0-c149-443e-a2d1-a30964775818/ovn-acl-logging/0.log"
Sep 15 16:41:26.332579 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:41:26.332544    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7" podUID="1b077164-aeea-45fb-ae73-dafe18b80859" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Sep 15 16:41:36.332552 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:41:36.332521    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7" podUID="1b077164-aeea-45fb-ae73-dafe18b80859" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Sep 15 16:41:46.333936 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:41:46.333911    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7"
Sep 15 16:41:58.247090 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:41:58.247060    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d"]
Sep 15 16:41:58.247530 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:41:58.247324    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d" podUID="71830964-3306-41dd-98d9-861abf40b42b" containerName="kube-rbac-proxy" containerID="cri-o://ebd96f52d195e0f0b85f6b10133547fc3e51695d9747fe6d2034569153472c41" gracePeriod=30
Sep 15 16:41:58.247530 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:41:58.247324    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d" podUID="71830964-3306-41dd-98d9-861abf40b42b" containerName="kserve-container" containerID="cri-o://9211f278b506c71f76ec4151937022716020cdf0680f7e18b57338fa9ae1caec" gracePeriod=30
Sep 15 16:41:58.658947 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:41:58.658880    2579 generic.go:356] "Generic (PLEG): container finished" podID="71830964-3306-41dd-98d9-861abf40b42b" containerID="ebd96f52d195e0f0b85f6b10133547fc3e51695d9747fe6d2034569153472c41" exitCode=2
Sep 15 16:41:58.658947 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:41:58.658937    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d" event={"ID":"71830964-3306-41dd-98d9-861abf40b42b","Type":"ContainerDied","Data":"ebd96f52d195e0f0b85f6b10133547fc3e51695d9747fe6d2034569153472c41"}
Sep 15 16:41:58.928147 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:41:58.928090    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d" podUID="71830964-3306-41dd-98d9-861abf40b42b" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.29:8643/healthz\": dial tcp 10.133.0.29:8643: connect: connection refused"
Sep 15 16:41:58.931447 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:41:58.931410    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d" podUID="71830964-3306-41dd-98d9-861abf40b42b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Sep 15 16:42:01.675538 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:01.675506    2579 generic.go:356] "Generic (PLEG): container finished" podID="71830964-3306-41dd-98d9-861abf40b42b" containerID="9211f278b506c71f76ec4151937022716020cdf0680f7e18b57338fa9ae1caec" exitCode=0
Sep 15 16:42:01.675940 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:01.675578    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d" event={"ID":"71830964-3306-41dd-98d9-861abf40b42b","Type":"ContainerDied","Data":"9211f278b506c71f76ec4151937022716020cdf0680f7e18b57338fa9ae1caec"}
Sep 15 16:42:01.785033 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:01.785011    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d"
Sep 15 16:42:01.938120 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:01.938069    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/71830964-3306-41dd-98d9-861abf40b42b-cabundle-cert\") pod \"71830964-3306-41dd-98d9-861abf40b42b\" (UID: \"71830964-3306-41dd-98d9-861abf40b42b\") "
Sep 15 16:42:01.938120 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:01.938104    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbxg6\" (UniqueName: \"kubernetes.io/projected/71830964-3306-41dd-98d9-861abf40b42b-kube-api-access-kbxg6\") pod \"71830964-3306-41dd-98d9-861abf40b42b\" (UID: \"71830964-3306-41dd-98d9-861abf40b42b\") "
Sep 15 16:42:01.938246 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:01.938126    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/71830964-3306-41dd-98d9-861abf40b42b-proxy-tls\") pod \"71830964-3306-41dd-98d9-861abf40b42b\" (UID: \"71830964-3306-41dd-98d9-861abf40b42b\") "
Sep 15 16:42:01.938246 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:01.938180    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/71830964-3306-41dd-98d9-861abf40b42b-kube-rbac-proxy-sar-config\") pod \"71830964-3306-41dd-98d9-861abf40b42b\" (UID: \"71830964-3306-41dd-98d9-861abf40b42b\") "
Sep 15 16:42:01.938246 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:01.938196    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/71830964-3306-41dd-98d9-861abf40b42b-kserve-provision-location\") pod \"71830964-3306-41dd-98d9-861abf40b42b\" (UID: \"71830964-3306-41dd-98d9-861abf40b42b\") "
Sep 15 16:42:01.938470 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:01.938444    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71830964-3306-41dd-98d9-861abf40b42b-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "71830964-3306-41dd-98d9-861abf40b42b" (UID: "71830964-3306-41dd-98d9-861abf40b42b"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:42:01.938528 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:01.938502    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71830964-3306-41dd-98d9-861abf40b42b-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "71830964-3306-41dd-98d9-861abf40b42b" (UID: "71830964-3306-41dd-98d9-861abf40b42b"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 16:42:01.938574 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:01.938529    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71830964-3306-41dd-98d9-861abf40b42b-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "71830964-3306-41dd-98d9-861abf40b42b" (UID: "71830964-3306-41dd-98d9-861abf40b42b"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:42:01.940392 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:01.940365    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71830964-3306-41dd-98d9-861abf40b42b-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "71830964-3306-41dd-98d9-861abf40b42b" (UID: "71830964-3306-41dd-98d9-861abf40b42b"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 16:42:01.940466 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:01.940450    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71830964-3306-41dd-98d9-861abf40b42b-kube-api-access-kbxg6" (OuterVolumeSpecName: "kube-api-access-kbxg6") pod "71830964-3306-41dd-98d9-861abf40b42b" (UID: "71830964-3306-41dd-98d9-861abf40b42b"). InnerVolumeSpecName "kube-api-access-kbxg6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 16:42:02.039467 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:02.039445    2579 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/71830964-3306-41dd-98d9-861abf40b42b-kube-rbac-proxy-sar-config\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:42:02.039575 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:02.039471    2579 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/71830964-3306-41dd-98d9-861abf40b42b-kserve-provision-location\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:42:02.039575 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:02.039486    2579 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/71830964-3306-41dd-98d9-861abf40b42b-cabundle-cert\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:42:02.039575 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:02.039499    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbxg6\" (UniqueName: \"kubernetes.io/projected/71830964-3306-41dd-98d9-861abf40b42b-kube-api-access-kbxg6\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:42:02.039575 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:02.039512    2579 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/71830964-3306-41dd-98d9-861abf40b42b-proxy-tls\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:42:02.681028 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:02.681006    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d" event={"ID":"71830964-3306-41dd-98d9-861abf40b42b","Type":"ContainerDied","Data":"742480abbd7f6df2387da593270f6b4d55ab2800fca9b8c031d3e960a7104f60"}
Sep 15 16:42:02.681326 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:02.681035    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d"
Sep 15 16:42:02.681326 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:02.681071    2579 scope.go:117] "RemoveContainer" containerID="ebd96f52d195e0f0b85f6b10133547fc3e51695d9747fe6d2034569153472c41"
Sep 15 16:42:02.689153 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:02.689138    2579 scope.go:117] "RemoveContainer" containerID="9211f278b506c71f76ec4151937022716020cdf0680f7e18b57338fa9ae1caec"
Sep 15 16:42:02.696441 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:02.696426    2579 scope.go:117] "RemoveContainer" containerID="69c901cc003c67b1b98f7df51628d650b9b633386070496c5829138f74ab6d02"
Sep 15 16:42:02.701952 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:02.701932    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d"]
Sep 15 16:42:02.705580 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:02.705562    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-6b76f97c96-kkd7d"]
Sep 15 16:42:03.512941 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:03.512902    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h"]
Sep 15 16:42:03.513252 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:03.513240    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71830964-3306-41dd-98d9-861abf40b42b" containerName="storage-initializer"
Sep 15 16:42:03.513303 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:03.513254    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="71830964-3306-41dd-98d9-861abf40b42b" containerName="storage-initializer"
Sep 15 16:42:03.513303 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:03.513269    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71830964-3306-41dd-98d9-861abf40b42b" containerName="kube-rbac-proxy"
Sep 15 16:42:03.513303 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:03.513275    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="71830964-3306-41dd-98d9-861abf40b42b" containerName="kube-rbac-proxy"
Sep 15 16:42:03.513303 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:03.513285    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71830964-3306-41dd-98d9-861abf40b42b" containerName="kserve-container"
Sep 15 16:42:03.513303 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:03.513291    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="71830964-3306-41dd-98d9-861abf40b42b" containerName="kserve-container"
Sep 15 16:42:03.513439 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:03.513341    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="71830964-3306-41dd-98d9-861abf40b42b" containerName="kube-rbac-proxy"
Sep 15 16:42:03.513439 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:03.513349    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="71830964-3306-41dd-98d9-861abf40b42b" containerName="kserve-container"
Sep 15 16:42:03.517972 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:03.517948    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h"
Sep 15 16:42:03.520273 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:03.520244    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-stop-predictor-serving-cert\""
Sep 15 16:42:03.520404 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:03.520349    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-stop-kube-rbac-proxy-sar-config\""
Sep 15 16:42:03.527239 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:03.527215    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h"]
Sep 15 16:42:03.653078 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:03.653057    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7e1f2fd9-3708-43e8-a4cb-393aa513210e-proxy-tls\") pod \"isvc-canary-stop-predictor-7f6f9c568d-s9n2h\" (UID: \"7e1f2fd9-3708-43e8-a4cb-393aa513210e\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h"
Sep 15 16:42:03.653214 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:03.653083    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7e1f2fd9-3708-43e8-a4cb-393aa513210e-kserve-provision-location\") pod \"isvc-canary-stop-predictor-7f6f9c568d-s9n2h\" (UID: \"7e1f2fd9-3708-43e8-a4cb-393aa513210e\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h"
Sep 15 16:42:03.653214 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:03.653109    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrbxx\" (UniqueName: \"kubernetes.io/projected/7e1f2fd9-3708-43e8-a4cb-393aa513210e-kube-api-access-vrbxx\") pod \"isvc-canary-stop-predictor-7f6f9c568d-s9n2h\" (UID: \"7e1f2fd9-3708-43e8-a4cb-393aa513210e\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h"
Sep 15 16:42:03.653214 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:03.653132    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7e1f2fd9-3708-43e8-a4cb-393aa513210e-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-7f6f9c568d-s9n2h\" (UID: \"7e1f2fd9-3708-43e8-a4cb-393aa513210e\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h"
Sep 15 16:42:03.653214 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:03.653183    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7e1f2fd9-3708-43e8-a4cb-393aa513210e-cabundle-cert\") pod \"isvc-canary-stop-predictor-7f6f9c568d-s9n2h\" (UID: \"7e1f2fd9-3708-43e8-a4cb-393aa513210e\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h"
Sep 15 16:42:03.753563 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:03.753535    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7e1f2fd9-3708-43e8-a4cb-393aa513210e-cabundle-cert\") pod \"isvc-canary-stop-predictor-7f6f9c568d-s9n2h\" (UID: \"7e1f2fd9-3708-43e8-a4cb-393aa513210e\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h"
Sep 15 16:42:03.753905 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:03.753630    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7e1f2fd9-3708-43e8-a4cb-393aa513210e-proxy-tls\") pod \"isvc-canary-stop-predictor-7f6f9c568d-s9n2h\" (UID: \"7e1f2fd9-3708-43e8-a4cb-393aa513210e\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h"
Sep 15 16:42:03.753905 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:03.753656    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7e1f2fd9-3708-43e8-a4cb-393aa513210e-kserve-provision-location\") pod \"isvc-canary-stop-predictor-7f6f9c568d-s9n2h\" (UID: \"7e1f2fd9-3708-43e8-a4cb-393aa513210e\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h"
Sep 15 16:42:03.753905 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:03.753694    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrbxx\" (UniqueName: \"kubernetes.io/projected/7e1f2fd9-3708-43e8-a4cb-393aa513210e-kube-api-access-vrbxx\") pod \"isvc-canary-stop-predictor-7f6f9c568d-s9n2h\" (UID: \"7e1f2fd9-3708-43e8-a4cb-393aa513210e\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h"
Sep 15 16:42:03.753905 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:42:03.753776    2579 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-canary-stop-predictor-serving-cert: secret "isvc-canary-stop-predictor-serving-cert" not found
Sep 15 16:42:03.753905 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:03.753823    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7e1f2fd9-3708-43e8-a4cb-393aa513210e-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-7f6f9c568d-s9n2h\" (UID: \"7e1f2fd9-3708-43e8-a4cb-393aa513210e\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h"
Sep 15 16:42:03.753905 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:42:03.753866    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e1f2fd9-3708-43e8-a4cb-393aa513210e-proxy-tls podName:7e1f2fd9-3708-43e8-a4cb-393aa513210e nodeName:}" failed. No retries permitted until 2026-09-15 16:42:04.25384677 +0000 UTC m=+940.883501111 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/7e1f2fd9-3708-43e8-a4cb-393aa513210e-proxy-tls") pod "isvc-canary-stop-predictor-7f6f9c568d-s9n2h" (UID: "7e1f2fd9-3708-43e8-a4cb-393aa513210e") : secret "isvc-canary-stop-predictor-serving-cert" not found
Sep 15 16:42:03.754151 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:03.754031    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7e1f2fd9-3708-43e8-a4cb-393aa513210e-kserve-provision-location\") pod \"isvc-canary-stop-predictor-7f6f9c568d-s9n2h\" (UID: \"7e1f2fd9-3708-43e8-a4cb-393aa513210e\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h"
Sep 15 16:42:03.754275 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:03.754255    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7e1f2fd9-3708-43e8-a4cb-393aa513210e-cabundle-cert\") pod \"isvc-canary-stop-predictor-7f6f9c568d-s9n2h\" (UID: \"7e1f2fd9-3708-43e8-a4cb-393aa513210e\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h"
Sep 15 16:42:03.754370 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:03.754355    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7e1f2fd9-3708-43e8-a4cb-393aa513210e-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-7f6f9c568d-s9n2h\" (UID: \"7e1f2fd9-3708-43e8-a4cb-393aa513210e\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h"
Sep 15 16:42:03.763988 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:03.763940    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrbxx\" (UniqueName: \"kubernetes.io/projected/7e1f2fd9-3708-43e8-a4cb-393aa513210e-kube-api-access-vrbxx\") pod \"isvc-canary-stop-predictor-7f6f9c568d-s9n2h\" (UID: \"7e1f2fd9-3708-43e8-a4cb-393aa513210e\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h"
Sep 15 16:42:03.910911 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:03.910882    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71830964-3306-41dd-98d9-861abf40b42b" path="/var/lib/kubelet/pods/71830964-3306-41dd-98d9-861abf40b42b/volumes"
Sep 15 16:42:04.256973 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:04.256946    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7e1f2fd9-3708-43e8-a4cb-393aa513210e-proxy-tls\") pod \"isvc-canary-stop-predictor-7f6f9c568d-s9n2h\" (UID: \"7e1f2fd9-3708-43e8-a4cb-393aa513210e\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h"
Sep 15 16:42:04.259245 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:04.259226    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7e1f2fd9-3708-43e8-a4cb-393aa513210e-proxy-tls\") pod \"isvc-canary-stop-predictor-7f6f9c568d-s9n2h\" (UID: \"7e1f2fd9-3708-43e8-a4cb-393aa513210e\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h"
Sep 15 16:42:04.428721 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:04.428692    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h"
Sep 15 16:42:04.697762 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:04.697739    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h"]
Sep 15 16:42:04.700426 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:42:04.700396    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e1f2fd9_3708_43e8_a4cb_393aa513210e.slice/crio-489a0e6b37c6e43416ab4856b5b7980f778e6d22472eecc6048a76bd47a63b4f WatchSource:0}: Error finding container 489a0e6b37c6e43416ab4856b5b7980f778e6d22472eecc6048a76bd47a63b4f: Status 404 returned error can't find the container with id 489a0e6b37c6e43416ab4856b5b7980f778e6d22472eecc6048a76bd47a63b4f
Sep 15 16:42:05.696274 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:05.696241    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h" event={"ID":"7e1f2fd9-3708-43e8-a4cb-393aa513210e","Type":"ContainerStarted","Data":"2c43487e86f6b0b5e7f5b30266b3c571aea77d9b44833349dd4d783b79380e22"}
Sep 15 16:42:05.696641 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:05.696301    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h" event={"ID":"7e1f2fd9-3708-43e8-a4cb-393aa513210e","Type":"ContainerStarted","Data":"489a0e6b37c6e43416ab4856b5b7980f778e6d22472eecc6048a76bd47a63b4f"}
Sep 15 16:42:09.713521 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:09.713495    2579 generic.go:356] "Generic (PLEG): container finished" podID="7e1f2fd9-3708-43e8-a4cb-393aa513210e" containerID="2c43487e86f6b0b5e7f5b30266b3c571aea77d9b44833349dd4d783b79380e22" exitCode=0
Sep 15 16:42:09.713834 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:09.713549    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h" event={"ID":"7e1f2fd9-3708-43e8-a4cb-393aa513210e","Type":"ContainerDied","Data":"2c43487e86f6b0b5e7f5b30266b3c571aea77d9b44833349dd4d783b79380e22"}
Sep 15 16:42:10.719980 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:10.719945    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h" event={"ID":"7e1f2fd9-3708-43e8-a4cb-393aa513210e","Type":"ContainerStarted","Data":"c1de44c2e830319dd0efbafc943410c0ec6aea22ff0109f2dda159ba9dd2468f"}
Sep 15 16:42:10.720285 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:10.720033    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h" event={"ID":"7e1f2fd9-3708-43e8-a4cb-393aa513210e","Type":"ContainerStarted","Data":"0e57693d8336f82fdeed6f8ca0e29322a097b4a80a9f2b10f6048d7de24b2dcf"}
Sep 15 16:42:10.720285 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:10.720227    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h"
Sep 15 16:42:10.740554 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:10.740511    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h" podStartSLOduration=3.584219562 podStartE2EDuration="7.740497683s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.156278121s" isStatefulPod=false podCreationTimestamp="2026-09-15 16:42:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:42:10.739484145 +0000 UTC m=+947.369138495" watchObservedRunningTime="2026-09-15 16:42:10.740497683 +0000 UTC m=+947.370152033"
Sep 15 16:42:11.724281 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:11.724238    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h"
Sep 15 16:42:11.725395 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:11.725363    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h" podUID="7e1f2fd9-3708-43e8-a4cb-393aa513210e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Sep 15 16:42:12.728179 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:12.728142    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h" podUID="7e1f2fd9-3708-43e8-a4cb-393aa513210e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Sep 15 16:42:17.732241 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:17.732211    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h"
Sep 15 16:42:17.732675 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:17.732626    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h" podUID="7e1f2fd9-3708-43e8-a4cb-393aa513210e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Sep 15 16:42:23.493852 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:23.493817    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7"]
Sep 15 16:42:23.494260 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:23.494039    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7" podUID="1b077164-aeea-45fb-ae73-dafe18b80859" containerName="kserve-container" containerID="cri-o://5873363d901ef97b51fdcc801ff5feeecbf06c6b353bba210059da921cab572c" gracePeriod=30
Sep 15 16:42:23.494260 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:23.494086    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7" podUID="1b077164-aeea-45fb-ae73-dafe18b80859" containerName="kube-rbac-proxy" containerID="cri-o://d08508d5d5be9923ea990b75305b4f753f84dc228366141c12a8b2d60bb6023d" gracePeriod=30
Sep 15 16:42:23.777389 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:23.777307    2579 generic.go:356] "Generic (PLEG): container finished" podID="1b077164-aeea-45fb-ae73-dafe18b80859" containerID="d08508d5d5be9923ea990b75305b4f753f84dc228366141c12a8b2d60bb6023d" exitCode=2
Sep 15 16:42:23.777389 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:23.777376    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7" event={"ID":"1b077164-aeea-45fb-ae73-dafe18b80859","Type":"ContainerDied","Data":"d08508d5d5be9923ea990b75305b4f753f84dc228366141c12a8b2d60bb6023d"}
Sep 15 16:42:26.328128 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:26.328090    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7" podUID="1b077164-aeea-45fb-ae73-dafe18b80859" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.30:8643/healthz\": dial tcp 10.133.0.30:8643: connect: connection refused"
Sep 15 16:42:26.333178 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:26.333151    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7" podUID="1b077164-aeea-45fb-ae73-dafe18b80859" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Sep 15 16:42:27.235775 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.235754    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7"
Sep 15 16:42:27.313928 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.313876    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1b077164-aeea-45fb-ae73-dafe18b80859-cabundle-cert\") pod \"1b077164-aeea-45fb-ae73-dafe18b80859\" (UID: \"1b077164-aeea-45fb-ae73-dafe18b80859\") "
Sep 15 16:42:27.314028 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.313957    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1b077164-aeea-45fb-ae73-dafe18b80859-kube-rbac-proxy-sar-config\") pod \"1b077164-aeea-45fb-ae73-dafe18b80859\" (UID: \"1b077164-aeea-45fb-ae73-dafe18b80859\") "
Sep 15 16:42:27.314028 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.313999    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t2q25\" (UniqueName: \"kubernetes.io/projected/1b077164-aeea-45fb-ae73-dafe18b80859-kube-api-access-t2q25\") pod \"1b077164-aeea-45fb-ae73-dafe18b80859\" (UID: \"1b077164-aeea-45fb-ae73-dafe18b80859\") "
Sep 15 16:42:27.314028 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.314023    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1b077164-aeea-45fb-ae73-dafe18b80859-kserve-provision-location\") pod \"1b077164-aeea-45fb-ae73-dafe18b80859\" (UID: \"1b077164-aeea-45fb-ae73-dafe18b80859\") "
Sep 15 16:42:27.314137 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.314038    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b077164-aeea-45fb-ae73-dafe18b80859-proxy-tls\") pod \"1b077164-aeea-45fb-ae73-dafe18b80859\" (UID: \"1b077164-aeea-45fb-ae73-dafe18b80859\") "
Sep 15 16:42:27.314275 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.314246    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b077164-aeea-45fb-ae73-dafe18b80859-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "1b077164-aeea-45fb-ae73-dafe18b80859" (UID: "1b077164-aeea-45fb-ae73-dafe18b80859"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:42:27.314346 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.314218    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b077164-aeea-45fb-ae73-dafe18b80859-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "1b077164-aeea-45fb-ae73-dafe18b80859" (UID: "1b077164-aeea-45fb-ae73-dafe18b80859"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:42:27.314346 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.314332    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b077164-aeea-45fb-ae73-dafe18b80859-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "1b077164-aeea-45fb-ae73-dafe18b80859" (UID: "1b077164-aeea-45fb-ae73-dafe18b80859"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 16:42:27.316095 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.316072    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b077164-aeea-45fb-ae73-dafe18b80859-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "1b077164-aeea-45fb-ae73-dafe18b80859" (UID: "1b077164-aeea-45fb-ae73-dafe18b80859"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 16:42:27.316168 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.316108    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b077164-aeea-45fb-ae73-dafe18b80859-kube-api-access-t2q25" (OuterVolumeSpecName: "kube-api-access-t2q25") pod "1b077164-aeea-45fb-ae73-dafe18b80859" (UID: "1b077164-aeea-45fb-ae73-dafe18b80859"). InnerVolumeSpecName "kube-api-access-t2q25". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 16:42:27.414813 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.414772    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2q25\" (UniqueName: \"kubernetes.io/projected/1b077164-aeea-45fb-ae73-dafe18b80859-kube-api-access-t2q25\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:42:27.414813 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.414812    2579 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1b077164-aeea-45fb-ae73-dafe18b80859-kserve-provision-location\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:42:27.415110 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.414825    2579 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b077164-aeea-45fb-ae73-dafe18b80859-proxy-tls\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:42:27.415110 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.414839    2579 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1b077164-aeea-45fb-ae73-dafe18b80859-cabundle-cert\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:42:27.415110 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.414850    2579 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1b077164-aeea-45fb-ae73-dafe18b80859-kube-rbac-proxy-sar-config\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:42:27.733058 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.733025    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h" podUID="7e1f2fd9-3708-43e8-a4cb-393aa513210e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Sep 15 16:42:27.797037 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.797013    2579 generic.go:356] "Generic (PLEG): container finished" podID="1b077164-aeea-45fb-ae73-dafe18b80859" containerID="5873363d901ef97b51fdcc801ff5feeecbf06c6b353bba210059da921cab572c" exitCode=0
Sep 15 16:42:27.797126 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.797075    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7"
Sep 15 16:42:27.797126 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.797080    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7" event={"ID":"1b077164-aeea-45fb-ae73-dafe18b80859","Type":"ContainerDied","Data":"5873363d901ef97b51fdcc801ff5feeecbf06c6b353bba210059da921cab572c"}
Sep 15 16:42:27.797217 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.797135    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7" event={"ID":"1b077164-aeea-45fb-ae73-dafe18b80859","Type":"ContainerDied","Data":"765c42bebd54b1ad943ddd337896377d4b068b276574bba1303347e3a5d43f3a"}
Sep 15 16:42:27.797217 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.797162    2579 scope.go:117] "RemoveContainer" containerID="d08508d5d5be9923ea990b75305b4f753f84dc228366141c12a8b2d60bb6023d"
Sep 15 16:42:27.807341 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.807319    2579 scope.go:117] "RemoveContainer" containerID="5873363d901ef97b51fdcc801ff5feeecbf06c6b353bba210059da921cab572c"
Sep 15 16:42:27.814461 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.814441    2579 scope.go:117] "RemoveContainer" containerID="77768bd3a6fc74fbfea86d1603c7583bf0b3ca4b2eb6fffa1a0a460fe68d2652"
Sep 15 16:42:27.820378 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.820354    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7"]
Sep 15 16:42:27.822382 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.822362    2579 scope.go:117] "RemoveContainer" containerID="d08508d5d5be9923ea990b75305b4f753f84dc228366141c12a8b2d60bb6023d"
Sep 15 16:42:27.822666 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:42:27.822649    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d08508d5d5be9923ea990b75305b4f753f84dc228366141c12a8b2d60bb6023d\": container with ID starting with d08508d5d5be9923ea990b75305b4f753f84dc228366141c12a8b2d60bb6023d not found: ID does not exist" containerID="d08508d5d5be9923ea990b75305b4f753f84dc228366141c12a8b2d60bb6023d"
Sep 15 16:42:27.822750 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.822675    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d08508d5d5be9923ea990b75305b4f753f84dc228366141c12a8b2d60bb6023d"} err="failed to get container status \"d08508d5d5be9923ea990b75305b4f753f84dc228366141c12a8b2d60bb6023d\": rpc error: code = NotFound desc = could not find container \"d08508d5d5be9923ea990b75305b4f753f84dc228366141c12a8b2d60bb6023d\": container with ID starting with d08508d5d5be9923ea990b75305b4f753f84dc228366141c12a8b2d60bb6023d not found: ID does not exist"
Sep 15 16:42:27.822750 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.822691    2579 scope.go:117] "RemoveContainer" containerID="5873363d901ef97b51fdcc801ff5feeecbf06c6b353bba210059da921cab572c"
Sep 15 16:42:27.822959 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:42:27.822941    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5873363d901ef97b51fdcc801ff5feeecbf06c6b353bba210059da921cab572c\": container with ID starting with 5873363d901ef97b51fdcc801ff5feeecbf06c6b353bba210059da921cab572c not found: ID does not exist" containerID="5873363d901ef97b51fdcc801ff5feeecbf06c6b353bba210059da921cab572c"
Sep 15 16:42:27.823014 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.822967    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5873363d901ef97b51fdcc801ff5feeecbf06c6b353bba210059da921cab572c"} err="failed to get container status \"5873363d901ef97b51fdcc801ff5feeecbf06c6b353bba210059da921cab572c\": rpc error: code = NotFound desc = could not find container \"5873363d901ef97b51fdcc801ff5feeecbf06c6b353bba210059da921cab572c\": container with ID starting with 5873363d901ef97b51fdcc801ff5feeecbf06c6b353bba210059da921cab572c not found: ID does not exist"
Sep 15 16:42:27.823014 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.822986    2579 scope.go:117] "RemoveContainer" containerID="77768bd3a6fc74fbfea86d1603c7583bf0b3ca4b2eb6fffa1a0a460fe68d2652"
Sep 15 16:42:27.823249 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:42:27.823231    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77768bd3a6fc74fbfea86d1603c7583bf0b3ca4b2eb6fffa1a0a460fe68d2652\": container with ID starting with 77768bd3a6fc74fbfea86d1603c7583bf0b3ca4b2eb6fffa1a0a460fe68d2652 not found: ID does not exist" containerID="77768bd3a6fc74fbfea86d1603c7583bf0b3ca4b2eb6fffa1a0a460fe68d2652"
Sep 15 16:42:27.823308 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.823252    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77768bd3a6fc74fbfea86d1603c7583bf0b3ca4b2eb6fffa1a0a460fe68d2652"} err="failed to get container status \"77768bd3a6fc74fbfea86d1603c7583bf0b3ca4b2eb6fffa1a0a460fe68d2652\": rpc error: code = NotFound desc = could not find container \"77768bd3a6fc74fbfea86d1603c7583bf0b3ca4b2eb6fffa1a0a460fe68d2652\": container with ID starting with 77768bd3a6fc74fbfea86d1603c7583bf0b3ca4b2eb6fffa1a0a460fe68d2652 not found: ID does not exist"
Sep 15 16:42:27.823308 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.823263    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-e2542-predictor-79d56dd4c7-zdmq7"]
Sep 15 16:42:27.908371 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:27.908348    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b077164-aeea-45fb-ae73-dafe18b80859" path="/var/lib/kubelet/pods/1b077164-aeea-45fb-ae73-dafe18b80859/volumes"
Sep 15 16:42:37.732775 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:37.732735    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h" podUID="7e1f2fd9-3708-43e8-a4cb-393aa513210e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Sep 15 16:42:47.732949 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:47.732864    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h" podUID="7e1f2fd9-3708-43e8-a4cb-393aa513210e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Sep 15 16:42:57.733451 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:42:57.733410    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h" podUID="7e1f2fd9-3708-43e8-a4cb-393aa513210e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Sep 15 16:43:07.733461 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:07.733427    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h" podUID="7e1f2fd9-3708-43e8-a4cb-393aa513210e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Sep 15 16:43:17.733964 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:17.733934    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h"
Sep 15 16:43:33.466174 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:33.466145    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h"]
Sep 15 16:43:33.466642 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:33.466382    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h" podUID="7e1f2fd9-3708-43e8-a4cb-393aa513210e" containerName="kserve-container" containerID="cri-o://0e57693d8336f82fdeed6f8ca0e29322a097b4a80a9f2b10f6048d7de24b2dcf" gracePeriod=30
Sep 15 16:43:33.466642 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:33.466445    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h" podUID="7e1f2fd9-3708-43e8-a4cb-393aa513210e" containerName="kube-rbac-proxy" containerID="cri-o://c1de44c2e830319dd0efbafc943410c0ec6aea22ff0109f2dda159ba9dd2468f" gracePeriod=30
Sep 15 16:43:34.071385 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:34.071355    2579 generic.go:356] "Generic (PLEG): container finished" podID="7e1f2fd9-3708-43e8-a4cb-393aa513210e" containerID="c1de44c2e830319dd0efbafc943410c0ec6aea22ff0109f2dda159ba9dd2468f" exitCode=2
Sep 15 16:43:34.071514 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:34.071393    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h" event={"ID":"7e1f2fd9-3708-43e8-a4cb-393aa513210e","Type":"ContainerDied","Data":"c1de44c2e830319dd0efbafc943410c0ec6aea22ff0109f2dda159ba9dd2468f"}
Sep 15 16:43:36.909572 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:36.909549    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h"
Sep 15 16:43:37.004181 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.004127    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7e1f2fd9-3708-43e8-a4cb-393aa513210e-kube-rbac-proxy-sar-config\") pod \"7e1f2fd9-3708-43e8-a4cb-393aa513210e\" (UID: \"7e1f2fd9-3708-43e8-a4cb-393aa513210e\") "
Sep 15 16:43:37.004181 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.004170    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7e1f2fd9-3708-43e8-a4cb-393aa513210e-kserve-provision-location\") pod \"7e1f2fd9-3708-43e8-a4cb-393aa513210e\" (UID: \"7e1f2fd9-3708-43e8-a4cb-393aa513210e\") "
Sep 15 16:43:37.004315 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.004199    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7e1f2fd9-3708-43e8-a4cb-393aa513210e-proxy-tls\") pod \"7e1f2fd9-3708-43e8-a4cb-393aa513210e\" (UID: \"7e1f2fd9-3708-43e8-a4cb-393aa513210e\") "
Sep 15 16:43:37.004315 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.004237    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7e1f2fd9-3708-43e8-a4cb-393aa513210e-cabundle-cert\") pod \"7e1f2fd9-3708-43e8-a4cb-393aa513210e\" (UID: \"7e1f2fd9-3708-43e8-a4cb-393aa513210e\") "
Sep 15 16:43:37.004394 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.004319    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrbxx\" (UniqueName: \"kubernetes.io/projected/7e1f2fd9-3708-43e8-a4cb-393aa513210e-kube-api-access-vrbxx\") pod \"7e1f2fd9-3708-43e8-a4cb-393aa513210e\" (UID: \"7e1f2fd9-3708-43e8-a4cb-393aa513210e\") "
Sep 15 16:43:37.004468 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.004446    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e1f2fd9-3708-43e8-a4cb-393aa513210e-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "7e1f2fd9-3708-43e8-a4cb-393aa513210e" (UID: "7e1f2fd9-3708-43e8-a4cb-393aa513210e"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 16:43:37.004524 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.004489    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e1f2fd9-3708-43e8-a4cb-393aa513210e-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "7e1f2fd9-3708-43e8-a4cb-393aa513210e" (UID: "7e1f2fd9-3708-43e8-a4cb-393aa513210e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:43:37.004644 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.004629    2579 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7e1f2fd9-3708-43e8-a4cb-393aa513210e-kserve-provision-location\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:43:37.004725 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.004645    2579 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7e1f2fd9-3708-43e8-a4cb-393aa513210e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:43:37.004725 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.004687    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e1f2fd9-3708-43e8-a4cb-393aa513210e-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "7e1f2fd9-3708-43e8-a4cb-393aa513210e" (UID: "7e1f2fd9-3708-43e8-a4cb-393aa513210e"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:43:37.006252 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.006233    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e1f2fd9-3708-43e8-a4cb-393aa513210e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "7e1f2fd9-3708-43e8-a4cb-393aa513210e" (UID: "7e1f2fd9-3708-43e8-a4cb-393aa513210e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 16:43:37.006462 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.006439    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e1f2fd9-3708-43e8-a4cb-393aa513210e-kube-api-access-vrbxx" (OuterVolumeSpecName: "kube-api-access-vrbxx") pod "7e1f2fd9-3708-43e8-a4cb-393aa513210e" (UID: "7e1f2fd9-3708-43e8-a4cb-393aa513210e"). InnerVolumeSpecName "kube-api-access-vrbxx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 16:43:37.085367 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.085340    2579 generic.go:356] "Generic (PLEG): container finished" podID="7e1f2fd9-3708-43e8-a4cb-393aa513210e" containerID="0e57693d8336f82fdeed6f8ca0e29322a097b4a80a9f2b10f6048d7de24b2dcf" exitCode=0
Sep 15 16:43:37.085450 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.085399    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h"
Sep 15 16:43:37.085450 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.085419    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h" event={"ID":"7e1f2fd9-3708-43e8-a4cb-393aa513210e","Type":"ContainerDied","Data":"0e57693d8336f82fdeed6f8ca0e29322a097b4a80a9f2b10f6048d7de24b2dcf"}
Sep 15 16:43:37.085535 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.085476    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h" event={"ID":"7e1f2fd9-3708-43e8-a4cb-393aa513210e","Type":"ContainerDied","Data":"489a0e6b37c6e43416ab4856b5b7980f778e6d22472eecc6048a76bd47a63b4f"}
Sep 15 16:43:37.085535 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.085501    2579 scope.go:117] "RemoveContainer" containerID="c1de44c2e830319dd0efbafc943410c0ec6aea22ff0109f2dda159ba9dd2468f"
Sep 15 16:43:37.093613 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.093598    2579 scope.go:117] "RemoveContainer" containerID="0e57693d8336f82fdeed6f8ca0e29322a097b4a80a9f2b10f6048d7de24b2dcf"
Sep 15 16:43:37.102874 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.102857    2579 scope.go:117] "RemoveContainer" containerID="2c43487e86f6b0b5e7f5b30266b3c571aea77d9b44833349dd4d783b79380e22"
Sep 15 16:43:37.105366 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.105347    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrbxx\" (UniqueName: \"kubernetes.io/projected/7e1f2fd9-3708-43e8-a4cb-393aa513210e-kube-api-access-vrbxx\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:43:37.105366 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.105365    2579 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7e1f2fd9-3708-43e8-a4cb-393aa513210e-proxy-tls\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:43:37.105515 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.105376    2579 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7e1f2fd9-3708-43e8-a4cb-393aa513210e-cabundle-cert\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:43:37.105662 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.105642    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h"]
Sep 15 16:43:37.110440 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.110424    2579 scope.go:117] "RemoveContainer" containerID="c1de44c2e830319dd0efbafc943410c0ec6aea22ff0109f2dda159ba9dd2468f"
Sep 15 16:43:37.110916 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:43:37.110854    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1de44c2e830319dd0efbafc943410c0ec6aea22ff0109f2dda159ba9dd2468f\": container with ID starting with c1de44c2e830319dd0efbafc943410c0ec6aea22ff0109f2dda159ba9dd2468f not found: ID does not exist" containerID="c1de44c2e830319dd0efbafc943410c0ec6aea22ff0109f2dda159ba9dd2468f"
Sep 15 16:43:37.110916 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.110887    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1de44c2e830319dd0efbafc943410c0ec6aea22ff0109f2dda159ba9dd2468f"} err="failed to get container status \"c1de44c2e830319dd0efbafc943410c0ec6aea22ff0109f2dda159ba9dd2468f\": rpc error: code = NotFound desc = could not find container \"c1de44c2e830319dd0efbafc943410c0ec6aea22ff0109f2dda159ba9dd2468f\": container with ID starting with c1de44c2e830319dd0efbafc943410c0ec6aea22ff0109f2dda159ba9dd2468f not found: ID does not exist"
Sep 15 16:43:37.110916 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.110910    2579 scope.go:117] "RemoveContainer" containerID="0e57693d8336f82fdeed6f8ca0e29322a097b4a80a9f2b10f6048d7de24b2dcf"
Sep 15 16:43:37.111245 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:43:37.111226    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e57693d8336f82fdeed6f8ca0e29322a097b4a80a9f2b10f6048d7de24b2dcf\": container with ID starting with 0e57693d8336f82fdeed6f8ca0e29322a097b4a80a9f2b10f6048d7de24b2dcf not found: ID does not exist" containerID="0e57693d8336f82fdeed6f8ca0e29322a097b4a80a9f2b10f6048d7de24b2dcf"
Sep 15 16:43:37.111324 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.111253    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e57693d8336f82fdeed6f8ca0e29322a097b4a80a9f2b10f6048d7de24b2dcf"} err="failed to get container status \"0e57693d8336f82fdeed6f8ca0e29322a097b4a80a9f2b10f6048d7de24b2dcf\": rpc error: code = NotFound desc = could not find container \"0e57693d8336f82fdeed6f8ca0e29322a097b4a80a9f2b10f6048d7de24b2dcf\": container with ID starting with 0e57693d8336f82fdeed6f8ca0e29322a097b4a80a9f2b10f6048d7de24b2dcf not found: ID does not exist"
Sep 15 16:43:37.111324 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.111274    2579 scope.go:117] "RemoveContainer" containerID="2c43487e86f6b0b5e7f5b30266b3c571aea77d9b44833349dd4d783b79380e22"
Sep 15 16:43:37.111537 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:43:37.111518    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c43487e86f6b0b5e7f5b30266b3c571aea77d9b44833349dd4d783b79380e22\": container with ID starting with 2c43487e86f6b0b5e7f5b30266b3c571aea77d9b44833349dd4d783b79380e22 not found: ID does not exist" containerID="2c43487e86f6b0b5e7f5b30266b3c571aea77d9b44833349dd4d783b79380e22"
Sep 15 16:43:37.111583 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.111544    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c43487e86f6b0b5e7f5b30266b3c571aea77d9b44833349dd4d783b79380e22"} err="failed to get container status \"2c43487e86f6b0b5e7f5b30266b3c571aea77d9b44833349dd4d783b79380e22\": rpc error: code = NotFound desc = could not find container \"2c43487e86f6b0b5e7f5b30266b3c571aea77d9b44833349dd4d783b79380e22\": container with ID starting with 2c43487e86f6b0b5e7f5b30266b3c571aea77d9b44833349dd4d783b79380e22 not found: ID does not exist"
Sep 15 16:43:37.112475 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.112457    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-predictor-7f6f9c568d-s9n2h"]
Sep 15 16:43:37.908735 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:43:37.908703    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e1f2fd9-3708-43e8-a4cb-393aa513210e" path="/var/lib/kubelet/pods/7e1f2fd9-3708-43e8-a4cb-393aa513210e/volumes"
Sep 15 16:44:23.953315 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:23.953291    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz"]
Sep 15 16:44:23.953640 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:23.953591    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b077164-aeea-45fb-ae73-dafe18b80859" containerName="kube-rbac-proxy"
Sep 15 16:44:23.953640 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:23.953600    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b077164-aeea-45fb-ae73-dafe18b80859" containerName="kube-rbac-proxy"
Sep 15 16:44:23.953640 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:23.953610    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e1f2fd9-3708-43e8-a4cb-393aa513210e" containerName="kserve-container"
Sep 15 16:44:23.953640 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:23.953615    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e1f2fd9-3708-43e8-a4cb-393aa513210e" containerName="kserve-container"
Sep 15 16:44:23.953640 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:23.953626    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b077164-aeea-45fb-ae73-dafe18b80859" containerName="kserve-container"
Sep 15 16:44:23.953640 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:23.953631    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b077164-aeea-45fb-ae73-dafe18b80859" containerName="kserve-container"
Sep 15 16:44:23.953640 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:23.953643    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b077164-aeea-45fb-ae73-dafe18b80859" containerName="storage-initializer"
Sep 15 16:44:23.953897 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:23.953648    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b077164-aeea-45fb-ae73-dafe18b80859" containerName="storage-initializer"
Sep 15 16:44:23.953897 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:23.953653    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e1f2fd9-3708-43e8-a4cb-393aa513210e" containerName="storage-initializer"
Sep 15 16:44:23.953897 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:23.953658    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e1f2fd9-3708-43e8-a4cb-393aa513210e" containerName="storage-initializer"
Sep 15 16:44:23.953897 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:23.953664    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e1f2fd9-3708-43e8-a4cb-393aa513210e" containerName="kube-rbac-proxy"
Sep 15 16:44:23.953897 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:23.953669    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e1f2fd9-3708-43e8-a4cb-393aa513210e" containerName="kube-rbac-proxy"
Sep 15 16:44:23.953897 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:23.953714    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b077164-aeea-45fb-ae73-dafe18b80859" containerName="kserve-container"
Sep 15 16:44:23.953897 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:23.953720    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e1f2fd9-3708-43e8-a4cb-393aa513210e" containerName="kube-rbac-proxy"
Sep 15 16:44:23.953897 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:23.953727    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e1f2fd9-3708-43e8-a4cb-393aa513210e" containerName="kserve-container"
Sep 15 16:44:23.953897 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:23.953734    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b077164-aeea-45fb-ae73-dafe18b80859" containerName="kube-rbac-proxy"
Sep 15 16:44:23.957968 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:23.957946    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz"
Sep 15 16:44:23.961808 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:23.961773    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"message-dumper-raw-ae4ee-predictor-serving-cert\""
Sep 15 16:44:23.962001 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:23.961872    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"message-dumper-raw-ae4ee-kube-rbac-proxy-sar-config\""
Sep 15 16:44:23.962475 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:23.962460    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-lqw4s\""
Sep 15 16:44:23.962930 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:23.962909    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 15 16:44:23.963028 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:23.962963    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 15 16:44:23.968141 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:23.968122    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz"]
Sep 15 16:44:24.017045 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:24.017009    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f0ffeb45-dfa1-4096-b1d6-8db7750f231f-proxy-tls\") pod \"message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz\" (UID: \"f0ffeb45-dfa1-4096-b1d6-8db7750f231f\") " pod="kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz"
Sep 15 16:44:24.017157 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:24.017092    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f0ffeb45-dfa1-4096-b1d6-8db7750f231f-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz\" (UID: \"f0ffeb45-dfa1-4096-b1d6-8db7750f231f\") " pod="kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz"
Sep 15 16:44:24.017157 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:24.017121    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24w6m\" (UniqueName: \"kubernetes.io/projected/f0ffeb45-dfa1-4096-b1d6-8db7750f231f-kube-api-access-24w6m\") pod \"message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz\" (UID: \"f0ffeb45-dfa1-4096-b1d6-8db7750f231f\") " pod="kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz"
Sep 15 16:44:24.117426 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:24.117407    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f0ffeb45-dfa1-4096-b1d6-8db7750f231f-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz\" (UID: \"f0ffeb45-dfa1-4096-b1d6-8db7750f231f\") " pod="kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz"
Sep 15 16:44:24.117536 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:24.117437    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24w6m\" (UniqueName: \"kubernetes.io/projected/f0ffeb45-dfa1-4096-b1d6-8db7750f231f-kube-api-access-24w6m\") pod \"message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz\" (UID: \"f0ffeb45-dfa1-4096-b1d6-8db7750f231f\") " pod="kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz"
Sep 15 16:44:24.117536 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:24.117478    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f0ffeb45-dfa1-4096-b1d6-8db7750f231f-proxy-tls\") pod \"message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz\" (UID: \"f0ffeb45-dfa1-4096-b1d6-8db7750f231f\") " pod="kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz"
Sep 15 16:44:24.118124 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:24.118101    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f0ffeb45-dfa1-4096-b1d6-8db7750f231f-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz\" (UID: \"f0ffeb45-dfa1-4096-b1d6-8db7750f231f\") " pod="kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz"
Sep 15 16:44:24.121288 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:24.121256    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f0ffeb45-dfa1-4096-b1d6-8db7750f231f-proxy-tls\") pod \"message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz\" (UID: \"f0ffeb45-dfa1-4096-b1d6-8db7750f231f\") " pod="kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz"
Sep 15 16:44:24.126083 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:24.126058    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24w6m\" (UniqueName: \"kubernetes.io/projected/f0ffeb45-dfa1-4096-b1d6-8db7750f231f-kube-api-access-24w6m\") pod \"message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz\" (UID: \"f0ffeb45-dfa1-4096-b1d6-8db7750f231f\") " pod="kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz"
Sep 15 16:44:24.268451 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:24.268433    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz"
Sep 15 16:44:24.535973 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:24.535954    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz"]
Sep 15 16:44:24.538249 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:44:24.538225    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0ffeb45_dfa1_4096_b1d6_8db7750f231f.slice/crio-1e37c970dc2b5521d3e8def16828fa0f5857e252a9919369ed8ee2c2c99e29c9 WatchSource:0}: Error finding container 1e37c970dc2b5521d3e8def16828fa0f5857e252a9919369ed8ee2c2c99e29c9: Status 404 returned error can't find the container with id 1e37c970dc2b5521d3e8def16828fa0f5857e252a9919369ed8ee2c2c99e29c9
Sep 15 16:44:24.540361 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:24.540343    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 16:44:25.268675 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:25.268647    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz" event={"ID":"f0ffeb45-dfa1-4096-b1d6-8db7750f231f","Type":"ContainerStarted","Data":"1e37c970dc2b5521d3e8def16828fa0f5857e252a9919369ed8ee2c2c99e29c9"}
Sep 15 16:44:26.275571 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:26.275496    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz" event={"ID":"f0ffeb45-dfa1-4096-b1d6-8db7750f231f","Type":"ContainerStarted","Data":"e369406442fed67a35422fb060c2b951ad1e04ea1ae34628eeaeab7f22c1a417"}
Sep 15 16:44:26.275571 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:26.275555    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz"
Sep 15 16:44:26.276016 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:26.275684    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz"
Sep 15 16:44:26.276016 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:26.275719    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz" event={"ID":"f0ffeb45-dfa1-4096-b1d6-8db7750f231f","Type":"ContainerStarted","Data":"0c84f0e85f2b611f7494afed52d3d755a7a54a8bcae88bb9273d30fb20c1a933"}
Sep 15 16:44:26.277161 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:26.277143    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz"
Sep 15 16:44:26.293477 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:26.293436    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz" podStartSLOduration=1.871345657 podStartE2EDuration="3.293421736s" totalImagesPullingTime="1.422076079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 16:44:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:44:26.292097235 +0000 UTC m=+1082.921751585" watchObservedRunningTime="2026-09-15 16:44:26.293421736 +0000 UTC m=+1082.923076086"
Sep 15 16:44:33.287617 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:44:33.287591    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz"
Sep 15 16:46:08.934472 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:08.934395    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz_f0ffeb45-dfa1-4096-b1d6-8db7750f231f/kserve-container/0.log"
Sep 15 16:46:09.125664 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.125635    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz"]
Sep 15 16:46:09.125921 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.125892    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz" podUID="f0ffeb45-dfa1-4096-b1d6-8db7750f231f" containerName="kserve-container" containerID="cri-o://0c84f0e85f2b611f7494afed52d3d755a7a54a8bcae88bb9273d30fb20c1a933" gracePeriod=30
Sep 15 16:46:09.126016 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.125918    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz" podUID="f0ffeb45-dfa1-4096-b1d6-8db7750f231f" containerName="kube-rbac-proxy" containerID="cri-o://e369406442fed67a35422fb060c2b951ad1e04ea1ae34628eeaeab7f22c1a417" gracePeriod=30
Sep 15 16:46:09.230678 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.230647    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx"]
Sep 15 16:46:09.235017 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.234997    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx"
Sep 15 16:46:09.237521 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.237499    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-scale-raw-d8f04-predictor-serving-cert\""
Sep 15 16:46:09.237616 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.237522    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Sep 15 16:46:09.237616 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.237550    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-scale-raw-d8f04-kube-rbac-proxy-sar-config\""
Sep 15 16:46:09.237768 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.237753    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Sep 15 16:46:09.245991 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.245969    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx"]
Sep 15 16:46:09.334270 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.334243    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/01c8e8e1-1248-4175-bb6d-2dd059baef67-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx\" (UID: \"01c8e8e1-1248-4175-bb6d-2dd059baef67\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx"
Sep 15 16:46:09.334409 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.334275    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/01c8e8e1-1248-4175-bb6d-2dd059baef67-kserve-provision-location\") pod \"isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx\" (UID: \"01c8e8e1-1248-4175-bb6d-2dd059baef67\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx"
Sep 15 16:46:09.334409 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.334393    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/01c8e8e1-1248-4175-bb6d-2dd059baef67-cabundle-cert\") pod \"isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx\" (UID: \"01c8e8e1-1248-4175-bb6d-2dd059baef67\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx"
Sep 15 16:46:09.334502 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.334466    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/01c8e8e1-1248-4175-bb6d-2dd059baef67-proxy-tls\") pod \"isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx\" (UID: \"01c8e8e1-1248-4175-bb6d-2dd059baef67\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx"
Sep 15 16:46:09.334502 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.334491    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fttmc\" (UniqueName: \"kubernetes.io/projected/01c8e8e1-1248-4175-bb6d-2dd059baef67-kube-api-access-fttmc\") pod \"isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx\" (UID: \"01c8e8e1-1248-4175-bb6d-2dd059baef67\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx"
Sep 15 16:46:09.376531 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.376512    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz"
Sep 15 16:46:09.436146 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.436118    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fttmc\" (UniqueName: \"kubernetes.io/projected/01c8e8e1-1248-4175-bb6d-2dd059baef67-kube-api-access-fttmc\") pod \"isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx\" (UID: \"01c8e8e1-1248-4175-bb6d-2dd059baef67\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx"
Sep 15 16:46:09.436257 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.436181    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/01c8e8e1-1248-4175-bb6d-2dd059baef67-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx\" (UID: \"01c8e8e1-1248-4175-bb6d-2dd059baef67\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx"
Sep 15 16:46:09.436257 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.436206    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/01c8e8e1-1248-4175-bb6d-2dd059baef67-kserve-provision-location\") pod \"isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx\" (UID: \"01c8e8e1-1248-4175-bb6d-2dd059baef67\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx"
Sep 15 16:46:09.436358 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.436274    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/01c8e8e1-1248-4175-bb6d-2dd059baef67-cabundle-cert\") pod \"isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx\" (UID: \"01c8e8e1-1248-4175-bb6d-2dd059baef67\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx"
Sep 15 16:46:09.436358 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.436330    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/01c8e8e1-1248-4175-bb6d-2dd059baef67-proxy-tls\") pod \"isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx\" (UID: \"01c8e8e1-1248-4175-bb6d-2dd059baef67\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx"
Sep 15 16:46:09.436616 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.436600    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/01c8e8e1-1248-4175-bb6d-2dd059baef67-kserve-provision-location\") pod \"isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx\" (UID: \"01c8e8e1-1248-4175-bb6d-2dd059baef67\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx"
Sep 15 16:46:09.436886 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.436866    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/01c8e8e1-1248-4175-bb6d-2dd059baef67-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx\" (UID: \"01c8e8e1-1248-4175-bb6d-2dd059baef67\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx"
Sep 15 16:46:09.436955 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.436938    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/01c8e8e1-1248-4175-bb6d-2dd059baef67-cabundle-cert\") pod \"isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx\" (UID: \"01c8e8e1-1248-4175-bb6d-2dd059baef67\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx"
Sep 15 16:46:09.438793 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.438758    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/01c8e8e1-1248-4175-bb6d-2dd059baef67-proxy-tls\") pod \"isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx\" (UID: \"01c8e8e1-1248-4175-bb6d-2dd059baef67\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx"
Sep 15 16:46:09.444229 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.444209    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fttmc\" (UniqueName: \"kubernetes.io/projected/01c8e8e1-1248-4175-bb6d-2dd059baef67-kube-api-access-fttmc\") pod \"isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx\" (UID: \"01c8e8e1-1248-4175-bb6d-2dd059baef67\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx"
Sep 15 16:46:09.537545 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.537497    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f0ffeb45-dfa1-4096-b1d6-8db7750f231f-kube-rbac-proxy-sar-config\") pod \"f0ffeb45-dfa1-4096-b1d6-8db7750f231f\" (UID: \"f0ffeb45-dfa1-4096-b1d6-8db7750f231f\") "
Sep 15 16:46:09.537618 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.537566    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f0ffeb45-dfa1-4096-b1d6-8db7750f231f-proxy-tls\") pod \"f0ffeb45-dfa1-4096-b1d6-8db7750f231f\" (UID: \"f0ffeb45-dfa1-4096-b1d6-8db7750f231f\") "
Sep 15 16:46:09.537618 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.537592    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24w6m\" (UniqueName: \"kubernetes.io/projected/f0ffeb45-dfa1-4096-b1d6-8db7750f231f-kube-api-access-24w6m\") pod \"f0ffeb45-dfa1-4096-b1d6-8db7750f231f\" (UID: \"f0ffeb45-dfa1-4096-b1d6-8db7750f231f\") "
Sep 15 16:46:09.537858 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.537827    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0ffeb45-dfa1-4096-b1d6-8db7750f231f-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "f0ffeb45-dfa1-4096-b1d6-8db7750f231f" (UID: "f0ffeb45-dfa1-4096-b1d6-8db7750f231f"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:46:09.539336 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.539316    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0ffeb45-dfa1-4096-b1d6-8db7750f231f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "f0ffeb45-dfa1-4096-b1d6-8db7750f231f" (UID: "f0ffeb45-dfa1-4096-b1d6-8db7750f231f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 16:46:09.539554 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.539528    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0ffeb45-dfa1-4096-b1d6-8db7750f231f-kube-api-access-24w6m" (OuterVolumeSpecName: "kube-api-access-24w6m") pod "f0ffeb45-dfa1-4096-b1d6-8db7750f231f" (UID: "f0ffeb45-dfa1-4096-b1d6-8db7750f231f"). InnerVolumeSpecName "kube-api-access-24w6m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 16:46:09.547706 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.547689    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx"
Sep 15 16:46:09.638329 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.638301    2579 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f0ffeb45-dfa1-4096-b1d6-8db7750f231f-proxy-tls\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:46:09.638329 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.638327    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24w6m\" (UniqueName: \"kubernetes.io/projected/f0ffeb45-dfa1-4096-b1d6-8db7750f231f-kube-api-access-24w6m\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:46:09.638474 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.638337    2579 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f0ffeb45-dfa1-4096-b1d6-8db7750f231f-kube-rbac-proxy-sar-config\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:46:09.678527 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.678504    2579 generic.go:356] "Generic (PLEG): container finished" podID="f0ffeb45-dfa1-4096-b1d6-8db7750f231f" containerID="e369406442fed67a35422fb060c2b951ad1e04ea1ae34628eeaeab7f22c1a417" exitCode=2
Sep 15 16:46:09.678527 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.678524    2579 generic.go:356] "Generic (PLEG): container finished" podID="f0ffeb45-dfa1-4096-b1d6-8db7750f231f" containerID="0c84f0e85f2b611f7494afed52d3d755a7a54a8bcae88bb9273d30fb20c1a933" exitCode=2
Sep 15 16:46:09.678662 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.678553    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz"
Sep 15 16:46:09.678662 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.678591    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz" event={"ID":"f0ffeb45-dfa1-4096-b1d6-8db7750f231f","Type":"ContainerDied","Data":"e369406442fed67a35422fb060c2b951ad1e04ea1ae34628eeaeab7f22c1a417"}
Sep 15 16:46:09.678662 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.678647    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz" event={"ID":"f0ffeb45-dfa1-4096-b1d6-8db7750f231f","Type":"ContainerDied","Data":"0c84f0e85f2b611f7494afed52d3d755a7a54a8bcae88bb9273d30fb20c1a933"}
Sep 15 16:46:09.678751 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.678666    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz" event={"ID":"f0ffeb45-dfa1-4096-b1d6-8db7750f231f","Type":"ContainerDied","Data":"1e37c970dc2b5521d3e8def16828fa0f5857e252a9919369ed8ee2c2c99e29c9"}
Sep 15 16:46:09.678751 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.678690    2579 scope.go:117] "RemoveContainer" containerID="e369406442fed67a35422fb060c2b951ad1e04ea1ae34628eeaeab7f22c1a417"
Sep 15 16:46:09.686860 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.686845    2579 scope.go:117] "RemoveContainer" containerID="0c84f0e85f2b611f7494afed52d3d755a7a54a8bcae88bb9273d30fb20c1a933"
Sep 15 16:46:09.693574 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.693561    2579 scope.go:117] "RemoveContainer" containerID="e369406442fed67a35422fb060c2b951ad1e04ea1ae34628eeaeab7f22c1a417"
Sep 15 16:46:09.693870 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:46:09.693837    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e369406442fed67a35422fb060c2b951ad1e04ea1ae34628eeaeab7f22c1a417\": container with ID starting with e369406442fed67a35422fb060c2b951ad1e04ea1ae34628eeaeab7f22c1a417 not found: ID does not exist" containerID="e369406442fed67a35422fb060c2b951ad1e04ea1ae34628eeaeab7f22c1a417"
Sep 15 16:46:09.693924 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.693884    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e369406442fed67a35422fb060c2b951ad1e04ea1ae34628eeaeab7f22c1a417"} err="failed to get container status \"e369406442fed67a35422fb060c2b951ad1e04ea1ae34628eeaeab7f22c1a417\": rpc error: code = NotFound desc = could not find container \"e369406442fed67a35422fb060c2b951ad1e04ea1ae34628eeaeab7f22c1a417\": container with ID starting with e369406442fed67a35422fb060c2b951ad1e04ea1ae34628eeaeab7f22c1a417 not found: ID does not exist"
Sep 15 16:46:09.693924 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.693901    2579 scope.go:117] "RemoveContainer" containerID="0c84f0e85f2b611f7494afed52d3d755a7a54a8bcae88bb9273d30fb20c1a933"
Sep 15 16:46:09.694138 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:46:09.694121    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c84f0e85f2b611f7494afed52d3d755a7a54a8bcae88bb9273d30fb20c1a933\": container with ID starting with 0c84f0e85f2b611f7494afed52d3d755a7a54a8bcae88bb9273d30fb20c1a933 not found: ID does not exist" containerID="0c84f0e85f2b611f7494afed52d3d755a7a54a8bcae88bb9273d30fb20c1a933"
Sep 15 16:46:09.694181 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.694145    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c84f0e85f2b611f7494afed52d3d755a7a54a8bcae88bb9273d30fb20c1a933"} err="failed to get container status \"0c84f0e85f2b611f7494afed52d3d755a7a54a8bcae88bb9273d30fb20c1a933\": rpc error: code = NotFound desc = could not find container \"0c84f0e85f2b611f7494afed52d3d755a7a54a8bcae88bb9273d30fb20c1a933\": container with ID starting with 0c84f0e85f2b611f7494afed52d3d755a7a54a8bcae88bb9273d30fb20c1a933 not found: ID does not exist"
Sep 15 16:46:09.694181 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.694159    2579 scope.go:117] "RemoveContainer" containerID="e369406442fed67a35422fb060c2b951ad1e04ea1ae34628eeaeab7f22c1a417"
Sep 15 16:46:09.694364 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.694344    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e369406442fed67a35422fb060c2b951ad1e04ea1ae34628eeaeab7f22c1a417"} err="failed to get container status \"e369406442fed67a35422fb060c2b951ad1e04ea1ae34628eeaeab7f22c1a417\": rpc error: code = NotFound desc = could not find container \"e369406442fed67a35422fb060c2b951ad1e04ea1ae34628eeaeab7f22c1a417\": container with ID starting with e369406442fed67a35422fb060c2b951ad1e04ea1ae34628eeaeab7f22c1a417 not found: ID does not exist"
Sep 15 16:46:09.694438 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.694368    2579 scope.go:117] "RemoveContainer" containerID="0c84f0e85f2b611f7494afed52d3d755a7a54a8bcae88bb9273d30fb20c1a933"
Sep 15 16:46:09.694585 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.694569    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c84f0e85f2b611f7494afed52d3d755a7a54a8bcae88bb9273d30fb20c1a933"} err="failed to get container status \"0c84f0e85f2b611f7494afed52d3d755a7a54a8bcae88bb9273d30fb20c1a933\": rpc error: code = NotFound desc = could not find container \"0c84f0e85f2b611f7494afed52d3d755a7a54a8bcae88bb9273d30fb20c1a933\": container with ID starting with 0c84f0e85f2b611f7494afed52d3d755a7a54a8bcae88bb9273d30fb20c1a933 not found: ID does not exist"
Sep 15 16:46:09.700261 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.700244    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz"]
Sep 15 16:46:09.706715 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.706693    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/message-dumper-raw-ae4ee-predictor-57c7b8756d-q25kz"]
Sep 15 16:46:09.838501 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.838447    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx"]
Sep 15 16:46:09.841273 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:46:09.841254    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01c8e8e1_1248_4175_bb6d_2dd059baef67.slice/crio-f6cd761acad1ab0b042cd5329d980e28b682020047ff52f799f3c6f8911d03bb WatchSource:0}: Error finding container f6cd761acad1ab0b042cd5329d980e28b682020047ff52f799f3c6f8911d03bb: Status 404 returned error can't find the container with id f6cd761acad1ab0b042cd5329d980e28b682020047ff52f799f3c6f8911d03bb
Sep 15 16:46:09.909118 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:09.909093    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0ffeb45-dfa1-4096-b1d6-8db7750f231f" path="/var/lib/kubelet/pods/f0ffeb45-dfa1-4096-b1d6-8db7750f231f/volumes"
Sep 15 16:46:10.684977 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:10.684946    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx" event={"ID":"01c8e8e1-1248-4175-bb6d-2dd059baef67","Type":"ContainerStarted","Data":"cad78b04838bc270936f5ce88fd669d91462d6266590fd358bf993726784efb2"}
Sep 15 16:46:10.685349 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:10.685008    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx" event={"ID":"01c8e8e1-1248-4175-bb6d-2dd059baef67","Type":"ContainerStarted","Data":"f6cd761acad1ab0b042cd5329d980e28b682020047ff52f799f3c6f8911d03bb"}
Sep 15 16:46:14.703700 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:14.703669    2579 generic.go:356] "Generic (PLEG): container finished" podID="01c8e8e1-1248-4175-bb6d-2dd059baef67" containerID="cad78b04838bc270936f5ce88fd669d91462d6266590fd358bf993726784efb2" exitCode=0
Sep 15 16:46:14.704031 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:14.703744    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx" event={"ID":"01c8e8e1-1248-4175-bb6d-2dd059baef67","Type":"ContainerDied","Data":"cad78b04838bc270936f5ce88fd669d91462d6266590fd358bf993726784efb2"}
Sep 15 16:46:15.715196 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:15.715160    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx" event={"ID":"01c8e8e1-1248-4175-bb6d-2dd059baef67","Type":"ContainerStarted","Data":"c7b3e199066b9659fd09fd370bf55b990dc97a2a2ade75efe8eb5a249f815eaa"}
Sep 15 16:46:15.715611 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:15.715257    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx" event={"ID":"01c8e8e1-1248-4175-bb6d-2dd059baef67","Type":"ContainerStarted","Data":"3d4df66d04c555c24895a4c5fa327538d0f5fdac17a57f13dea10a164bdf5959"}
Sep 15 16:46:15.715611 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:15.715467    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx"
Sep 15 16:46:15.733651 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:15.733607    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx" podStartSLOduration=2.947703031 podStartE2EDuration="6.7335967s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.785893669s" isStatefulPod=false podCreationTimestamp="2026-09-15 16:46:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:46:15.731355349 +0000 UTC m=+1192.361009700" watchObservedRunningTime="2026-09-15 16:46:15.7335967 +0000 UTC m=+1192.363251050"
Sep 15 16:46:16.719174 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:16.719149    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx"
Sep 15 16:46:16.720435 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:16.720403    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx" podUID="01c8e8e1-1248-4175-bb6d-2dd059baef67" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Sep 15 16:46:17.723599 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:17.723565    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx" podUID="01c8e8e1-1248-4175-bb6d-2dd059baef67" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Sep 15 16:46:22.727995 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:22.727966    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx"
Sep 15 16:46:22.728433 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:22.728411    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx" podUID="01c8e8e1-1248-4175-bb6d-2dd059baef67" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Sep 15 16:46:23.890091 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:23.890066    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4rdn_44ca88f0-c149-443e-a2d1-a30964775818/ovn-acl-logging/0.log"
Sep 15 16:46:23.892418 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:23.892399    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4rdn_44ca88f0-c149-443e-a2d1-a30964775818/ovn-acl-logging/0.log"
Sep 15 16:46:32.728399 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:32.728357    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx" podUID="01c8e8e1-1248-4175-bb6d-2dd059baef67" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Sep 15 16:46:42.728934 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:42.728894    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx" podUID="01c8e8e1-1248-4175-bb6d-2dd059baef67" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Sep 15 16:46:52.728552 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:46:52.728514    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx" podUID="01c8e8e1-1248-4175-bb6d-2dd059baef67" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Sep 15 16:47:02.728716 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:47:02.728682    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx" podUID="01c8e8e1-1248-4175-bb6d-2dd059baef67" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Sep 15 16:47:12.728380 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:47:12.728340    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx" podUID="01c8e8e1-1248-4175-bb6d-2dd059baef67" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Sep 15 16:47:22.728657 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:47:22.728572    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx" podUID="01c8e8e1-1248-4175-bb6d-2dd059baef67" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Sep 15 16:47:29.903530 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:47:29.903486    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx" podUID="01c8e8e1-1248-4175-bb6d-2dd059baef67" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Sep 15 16:47:39.904204 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:47:39.904159    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx" podUID="01c8e8e1-1248-4175-bb6d-2dd059baef67" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Sep 15 16:47:49.904193 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:47:49.904139    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx" podUID="01c8e8e1-1248-4175-bb6d-2dd059baef67" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Sep 15 16:47:59.904241 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:47:59.904197    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx" podUID="01c8e8e1-1248-4175-bb6d-2dd059baef67" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Sep 15 16:48:09.904103 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:09.904053    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx" podUID="01c8e8e1-1248-4175-bb6d-2dd059baef67" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Sep 15 16:48:19.904473 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:19.904426    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx" podUID="01c8e8e1-1248-4175-bb6d-2dd059baef67" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Sep 15 16:48:29.908451 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:29.908415    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx"
Sep 15 16:48:39.273370 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:39.273334    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx"]
Sep 15 16:48:39.274091 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:39.273571    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx" podUID="01c8e8e1-1248-4175-bb6d-2dd059baef67" containerName="kserve-container" containerID="cri-o://3d4df66d04c555c24895a4c5fa327538d0f5fdac17a57f13dea10a164bdf5959" gracePeriod=30
Sep 15 16:48:39.274091 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:39.273617    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx" podUID="01c8e8e1-1248-4175-bb6d-2dd059baef67" containerName="kube-rbac-proxy" containerID="cri-o://c7b3e199066b9659fd09fd370bf55b990dc97a2a2ade75efe8eb5a249f815eaa" gracePeriod=30
Sep 15 16:48:39.903646 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:39.903599    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx" podUID="01c8e8e1-1248-4175-bb6d-2dd059baef67" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Sep 15 16:48:40.311094 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:40.311068    2579 generic.go:356] "Generic (PLEG): container finished" podID="01c8e8e1-1248-4175-bb6d-2dd059baef67" containerID="c7b3e199066b9659fd09fd370bf55b990dc97a2a2ade75efe8eb5a249f815eaa" exitCode=2
Sep 15 16:48:40.311392 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:40.311130    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx" event={"ID":"01c8e8e1-1248-4175-bb6d-2dd059baef67","Type":"ContainerDied","Data":"c7b3e199066b9659fd09fd370bf55b990dc97a2a2ade75efe8eb5a249f815eaa"}
Sep 15 16:48:42.724081 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:42.724037    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx" podUID="01c8e8e1-1248-4175-bb6d-2dd059baef67" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.33:8643/healthz\": dial tcp 10.133.0.33:8643: connect: connection refused"
Sep 15 16:48:44.550114 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:44.550075    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj"]
Sep 15 16:48:44.550537 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:44.550450    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0ffeb45-dfa1-4096-b1d6-8db7750f231f" containerName="kube-rbac-proxy"
Sep 15 16:48:44.550537 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:44.550463    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0ffeb45-dfa1-4096-b1d6-8db7750f231f" containerName="kube-rbac-proxy"
Sep 15 16:48:44.550537 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:44.550477    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0ffeb45-dfa1-4096-b1d6-8db7750f231f" containerName="kserve-container"
Sep 15 16:48:44.550537 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:44.550482    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0ffeb45-dfa1-4096-b1d6-8db7750f231f" containerName="kserve-container"
Sep 15 16:48:44.550537 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:44.550538    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0ffeb45-dfa1-4096-b1d6-8db7750f231f" containerName="kserve-container"
Sep 15 16:48:44.550774 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:44.550546    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0ffeb45-dfa1-4096-b1d6-8db7750f231f" containerName="kube-rbac-proxy"
Sep 15 16:48:44.553857 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:44.553838    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj"
Sep 15 16:48:44.556187 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:44.556165    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-create-predictor-serving-cert\""
Sep 15 16:48:44.556291 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:44.556165    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-create-kube-rbac-proxy-sar-config\""
Sep 15 16:48:44.562963 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:44.562942    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj"]
Sep 15 16:48:44.657600 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:44.657577    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d701f09c-42b4-47d8-8ee4-23ebcd42dd54-kserve-provision-location\") pod \"isvc-canary-create-predictor-7d5b67cc88-gzdsj\" (UID: \"d701f09c-42b4-47d8-8ee4-23ebcd42dd54\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj"
Sep 15 16:48:44.657721 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:44.657658    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d701f09c-42b4-47d8-8ee4-23ebcd42dd54-cabundle-cert\") pod \"isvc-canary-create-predictor-7d5b67cc88-gzdsj\" (UID: \"d701f09c-42b4-47d8-8ee4-23ebcd42dd54\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj"
Sep 15 16:48:44.657773 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:44.657729    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d701f09c-42b4-47d8-8ee4-23ebcd42dd54-proxy-tls\") pod \"isvc-canary-create-predictor-7d5b67cc88-gzdsj\" (UID: \"d701f09c-42b4-47d8-8ee4-23ebcd42dd54\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj"
Sep 15 16:48:44.657859 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:44.657831    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d701f09c-42b4-47d8-8ee4-23ebcd42dd54-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-7d5b67cc88-gzdsj\" (UID: \"d701f09c-42b4-47d8-8ee4-23ebcd42dd54\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj"
Sep 15 16:48:44.657900 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:44.657871    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dvwz\" (UniqueName: \"kubernetes.io/projected/d701f09c-42b4-47d8-8ee4-23ebcd42dd54-kube-api-access-7dvwz\") pod \"isvc-canary-create-predictor-7d5b67cc88-gzdsj\" (UID: \"d701f09c-42b4-47d8-8ee4-23ebcd42dd54\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj"
Sep 15 16:48:44.758585 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:44.758557    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dvwz\" (UniqueName: \"kubernetes.io/projected/d701f09c-42b4-47d8-8ee4-23ebcd42dd54-kube-api-access-7dvwz\") pod \"isvc-canary-create-predictor-7d5b67cc88-gzdsj\" (UID: \"d701f09c-42b4-47d8-8ee4-23ebcd42dd54\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj"
Sep 15 16:48:44.758585 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:44.758589    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d701f09c-42b4-47d8-8ee4-23ebcd42dd54-kserve-provision-location\") pod \"isvc-canary-create-predictor-7d5b67cc88-gzdsj\" (UID: \"d701f09c-42b4-47d8-8ee4-23ebcd42dd54\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj"
Sep 15 16:48:44.758773 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:44.758642    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d701f09c-42b4-47d8-8ee4-23ebcd42dd54-cabundle-cert\") pod \"isvc-canary-create-predictor-7d5b67cc88-gzdsj\" (UID: \"d701f09c-42b4-47d8-8ee4-23ebcd42dd54\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj"
Sep 15 16:48:44.758773 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:44.758670    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d701f09c-42b4-47d8-8ee4-23ebcd42dd54-proxy-tls\") pod \"isvc-canary-create-predictor-7d5b67cc88-gzdsj\" (UID: \"d701f09c-42b4-47d8-8ee4-23ebcd42dd54\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj"
Sep 15 16:48:44.758773 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:44.758707    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d701f09c-42b4-47d8-8ee4-23ebcd42dd54-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-7d5b67cc88-gzdsj\" (UID: \"d701f09c-42b4-47d8-8ee4-23ebcd42dd54\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj"
Sep 15 16:48:44.759044 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:44.759026    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d701f09c-42b4-47d8-8ee4-23ebcd42dd54-kserve-provision-location\") pod \"isvc-canary-create-predictor-7d5b67cc88-gzdsj\" (UID: \"d701f09c-42b4-47d8-8ee4-23ebcd42dd54\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj"
Sep 15 16:48:44.759388 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:44.759371    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d701f09c-42b4-47d8-8ee4-23ebcd42dd54-cabundle-cert\") pod \"isvc-canary-create-predictor-7d5b67cc88-gzdsj\" (UID: \"d701f09c-42b4-47d8-8ee4-23ebcd42dd54\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj"
Sep 15 16:48:44.759434 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:44.759379    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d701f09c-42b4-47d8-8ee4-23ebcd42dd54-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-7d5b67cc88-gzdsj\" (UID: \"d701f09c-42b4-47d8-8ee4-23ebcd42dd54\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj"
Sep 15 16:48:44.761238 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:44.761218    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d701f09c-42b4-47d8-8ee4-23ebcd42dd54-proxy-tls\") pod \"isvc-canary-create-predictor-7d5b67cc88-gzdsj\" (UID: \"d701f09c-42b4-47d8-8ee4-23ebcd42dd54\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj"
Sep 15 16:48:44.765899 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:44.765882    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dvwz\" (UniqueName: \"kubernetes.io/projected/d701f09c-42b4-47d8-8ee4-23ebcd42dd54-kube-api-access-7dvwz\") pod \"isvc-canary-create-predictor-7d5b67cc88-gzdsj\" (UID: \"d701f09c-42b4-47d8-8ee4-23ebcd42dd54\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj"
Sep 15 16:48:44.866581 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:44.866527    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj"
Sep 15 16:48:45.135259 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:45.135235    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj"]
Sep 15 16:48:45.137223 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:48:45.137197    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd701f09c_42b4_47d8_8ee4_23ebcd42dd54.slice/crio-b58d47a19e666da532ce3524ab8fbf430dbb251034c07935279f7cdddbb8fe2d WatchSource:0}: Error finding container b58d47a19e666da532ce3524ab8fbf430dbb251034c07935279f7cdddbb8fe2d: Status 404 returned error can't find the container with id b58d47a19e666da532ce3524ab8fbf430dbb251034c07935279f7cdddbb8fe2d
Sep 15 16:48:45.332389 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:45.332357    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj" event={"ID":"d701f09c-42b4-47d8-8ee4-23ebcd42dd54","Type":"ContainerStarted","Data":"59a18fd5805445464cec4b04e9399078e7ab5c77e007b33e1d909f1de9c69a95"}
Sep 15 16:48:45.332524 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:45.332408    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj" event={"ID":"d701f09c-42b4-47d8-8ee4-23ebcd42dd54","Type":"ContainerStarted","Data":"b58d47a19e666da532ce3524ab8fbf430dbb251034c07935279f7cdddbb8fe2d"}
Sep 15 16:48:47.310256 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.310235    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx"
Sep 15 16:48:47.341874 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.341846    2579 generic.go:356] "Generic (PLEG): container finished" podID="01c8e8e1-1248-4175-bb6d-2dd059baef67" containerID="3d4df66d04c555c24895a4c5fa327538d0f5fdac17a57f13dea10a164bdf5959" exitCode=0
Sep 15 16:48:47.342012 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.341901    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx"
Sep 15 16:48:47.342012 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.341926    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx" event={"ID":"01c8e8e1-1248-4175-bb6d-2dd059baef67","Type":"ContainerDied","Data":"3d4df66d04c555c24895a4c5fa327538d0f5fdac17a57f13dea10a164bdf5959"}
Sep 15 16:48:47.342099 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.342009    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx" event={"ID":"01c8e8e1-1248-4175-bb6d-2dd059baef67","Type":"ContainerDied","Data":"f6cd761acad1ab0b042cd5329d980e28b682020047ff52f799f3c6f8911d03bb"}
Sep 15 16:48:47.342099 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.342052    2579 scope.go:117] "RemoveContainer" containerID="c7b3e199066b9659fd09fd370bf55b990dc97a2a2ade75efe8eb5a249f815eaa"
Sep 15 16:48:47.350742 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.350723    2579 scope.go:117] "RemoveContainer" containerID="3d4df66d04c555c24895a4c5fa327538d0f5fdac17a57f13dea10a164bdf5959"
Sep 15 16:48:47.360423 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.360407    2579 scope.go:117] "RemoveContainer" containerID="cad78b04838bc270936f5ce88fd669d91462d6266590fd358bf993726784efb2"
Sep 15 16:48:47.367324 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.367311    2579 scope.go:117] "RemoveContainer" containerID="c7b3e199066b9659fd09fd370bf55b990dc97a2a2ade75efe8eb5a249f815eaa"
Sep 15 16:48:47.367591 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:48:47.367570    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7b3e199066b9659fd09fd370bf55b990dc97a2a2ade75efe8eb5a249f815eaa\": container with ID starting with c7b3e199066b9659fd09fd370bf55b990dc97a2a2ade75efe8eb5a249f815eaa not found: ID does not exist" containerID="c7b3e199066b9659fd09fd370bf55b990dc97a2a2ade75efe8eb5a249f815eaa"
Sep 15 16:48:47.367661 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.367602    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7b3e199066b9659fd09fd370bf55b990dc97a2a2ade75efe8eb5a249f815eaa"} err="failed to get container status \"c7b3e199066b9659fd09fd370bf55b990dc97a2a2ade75efe8eb5a249f815eaa\": rpc error: code = NotFound desc = could not find container \"c7b3e199066b9659fd09fd370bf55b990dc97a2a2ade75efe8eb5a249f815eaa\": container with ID starting with c7b3e199066b9659fd09fd370bf55b990dc97a2a2ade75efe8eb5a249f815eaa not found: ID does not exist"
Sep 15 16:48:47.367661 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.367617    2579 scope.go:117] "RemoveContainer" containerID="3d4df66d04c555c24895a4c5fa327538d0f5fdac17a57f13dea10a164bdf5959"
Sep 15 16:48:47.367888 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:48:47.367873    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d4df66d04c555c24895a4c5fa327538d0f5fdac17a57f13dea10a164bdf5959\": container with ID starting with 3d4df66d04c555c24895a4c5fa327538d0f5fdac17a57f13dea10a164bdf5959 not found: ID does not exist" containerID="3d4df66d04c555c24895a4c5fa327538d0f5fdac17a57f13dea10a164bdf5959"
Sep 15 16:48:47.367943 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.367890    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d4df66d04c555c24895a4c5fa327538d0f5fdac17a57f13dea10a164bdf5959"} err="failed to get container status \"3d4df66d04c555c24895a4c5fa327538d0f5fdac17a57f13dea10a164bdf5959\": rpc error: code = NotFound desc = could not find container \"3d4df66d04c555c24895a4c5fa327538d0f5fdac17a57f13dea10a164bdf5959\": container with ID starting with 3d4df66d04c555c24895a4c5fa327538d0f5fdac17a57f13dea10a164bdf5959 not found: ID does not exist"
Sep 15 16:48:47.367943 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.367903    2579 scope.go:117] "RemoveContainer" containerID="cad78b04838bc270936f5ce88fd669d91462d6266590fd358bf993726784efb2"
Sep 15 16:48:47.368189 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:48:47.368164    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cad78b04838bc270936f5ce88fd669d91462d6266590fd358bf993726784efb2\": container with ID starting with cad78b04838bc270936f5ce88fd669d91462d6266590fd358bf993726784efb2 not found: ID does not exist" containerID="cad78b04838bc270936f5ce88fd669d91462d6266590fd358bf993726784efb2"
Sep 15 16:48:47.368269 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.368192    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cad78b04838bc270936f5ce88fd669d91462d6266590fd358bf993726784efb2"} err="failed to get container status \"cad78b04838bc270936f5ce88fd669d91462d6266590fd358bf993726784efb2\": rpc error: code = NotFound desc = could not find container \"cad78b04838bc270936f5ce88fd669d91462d6266590fd358bf993726784efb2\": container with ID starting with cad78b04838bc270936f5ce88fd669d91462d6266590fd358bf993726784efb2 not found: ID does not exist"
Sep 15 16:48:47.377464 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.377446    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/01c8e8e1-1248-4175-bb6d-2dd059baef67-kube-rbac-proxy-sar-config\") pod \"01c8e8e1-1248-4175-bb6d-2dd059baef67\" (UID: \"01c8e8e1-1248-4175-bb6d-2dd059baef67\") "
Sep 15 16:48:47.377530 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.377492    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/01c8e8e1-1248-4175-bb6d-2dd059baef67-cabundle-cert\") pod \"01c8e8e1-1248-4175-bb6d-2dd059baef67\" (UID: \"01c8e8e1-1248-4175-bb6d-2dd059baef67\") "
Sep 15 16:48:47.377530 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.377514    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/01c8e8e1-1248-4175-bb6d-2dd059baef67-proxy-tls\") pod \"01c8e8e1-1248-4175-bb6d-2dd059baef67\" (UID: \"01c8e8e1-1248-4175-bb6d-2dd059baef67\") "
Sep 15 16:48:47.377598 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.377552    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fttmc\" (UniqueName: \"kubernetes.io/projected/01c8e8e1-1248-4175-bb6d-2dd059baef67-kube-api-access-fttmc\") pod \"01c8e8e1-1248-4175-bb6d-2dd059baef67\" (UID: \"01c8e8e1-1248-4175-bb6d-2dd059baef67\") "
Sep 15 16:48:47.377598 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.377577    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/01c8e8e1-1248-4175-bb6d-2dd059baef67-kserve-provision-location\") pod \"01c8e8e1-1248-4175-bb6d-2dd059baef67\" (UID: \"01c8e8e1-1248-4175-bb6d-2dd059baef67\") "
Sep 15 16:48:47.377826 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.377797    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01c8e8e1-1248-4175-bb6d-2dd059baef67-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "01c8e8e1-1248-4175-bb6d-2dd059baef67" (UID: "01c8e8e1-1248-4175-bb6d-2dd059baef67"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:48:47.377952 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.377848    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01c8e8e1-1248-4175-bb6d-2dd059baef67-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "01c8e8e1-1248-4175-bb6d-2dd059baef67" (UID: "01c8e8e1-1248-4175-bb6d-2dd059baef67"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:48:47.378020 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.377984    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01c8e8e1-1248-4175-bb6d-2dd059baef67-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "01c8e8e1-1248-4175-bb6d-2dd059baef67" (UID: "01c8e8e1-1248-4175-bb6d-2dd059baef67"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 16:48:47.379557 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.379535    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01c8e8e1-1248-4175-bb6d-2dd059baef67-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "01c8e8e1-1248-4175-bb6d-2dd059baef67" (UID: "01c8e8e1-1248-4175-bb6d-2dd059baef67"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 16:48:47.379666 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.379647    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01c8e8e1-1248-4175-bb6d-2dd059baef67-kube-api-access-fttmc" (OuterVolumeSpecName: "kube-api-access-fttmc") pod "01c8e8e1-1248-4175-bb6d-2dd059baef67" (UID: "01c8e8e1-1248-4175-bb6d-2dd059baef67"). InnerVolumeSpecName "kube-api-access-fttmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 16:48:47.478055 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.478035    2579 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/01c8e8e1-1248-4175-bb6d-2dd059baef67-kube-rbac-proxy-sar-config\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:48:47.478055 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.478055    2579 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/01c8e8e1-1248-4175-bb6d-2dd059baef67-cabundle-cert\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:48:47.478160 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.478064    2579 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/01c8e8e1-1248-4175-bb6d-2dd059baef67-proxy-tls\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:48:47.478160 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.478072    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fttmc\" (UniqueName: \"kubernetes.io/projected/01c8e8e1-1248-4175-bb6d-2dd059baef67-kube-api-access-fttmc\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:48:47.478160 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.478080    2579 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/01c8e8e1-1248-4175-bb6d-2dd059baef67-kserve-provision-location\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:48:47.663480 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.663452    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx"]
Sep 15 16:48:47.665112 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.665091    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-scale-raw-d8f04-predictor-bf4d886f5-jklcx"]
Sep 15 16:48:47.909116 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:47.909066    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01c8e8e1-1248-4175-bb6d-2dd059baef67" path="/var/lib/kubelet/pods/01c8e8e1-1248-4175-bb6d-2dd059baef67/volumes"
Sep 15 16:48:49.352022 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:49.351993    2579 generic.go:356] "Generic (PLEG): container finished" podID="d701f09c-42b4-47d8-8ee4-23ebcd42dd54" containerID="59a18fd5805445464cec4b04e9399078e7ab5c77e007b33e1d909f1de9c69a95" exitCode=0
Sep 15 16:48:49.352343 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:49.352066    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj" event={"ID":"d701f09c-42b4-47d8-8ee4-23ebcd42dd54","Type":"ContainerDied","Data":"59a18fd5805445464cec4b04e9399078e7ab5c77e007b33e1d909f1de9c69a95"}
Sep 15 16:48:50.358149 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:50.358107    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj" event={"ID":"d701f09c-42b4-47d8-8ee4-23ebcd42dd54","Type":"ContainerStarted","Data":"78d45c05f310877abb2fe8ce17aa0fec53a98bb80b18d6c9cbea6a46a1538c7d"}
Sep 15 16:48:50.358563 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:50.358176    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj" event={"ID":"d701f09c-42b4-47d8-8ee4-23ebcd42dd54","Type":"ContainerStarted","Data":"3fbad23d92166cf78f63d32f6c1cb57a4310b8d2513f5a35b5d7dafc5affd5bc"}
Sep 15 16:48:50.358563 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:50.358422    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj"
Sep 15 16:48:50.358563 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:50.358554    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj"
Sep 15 16:48:50.359770 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:50.359743    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj" podUID="d701f09c-42b4-47d8-8ee4-23ebcd42dd54" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Sep 15 16:48:50.379905 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:50.379867    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj" podStartSLOduration=2.662839578 podStartE2EDuration="6.37985406s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.717014482s" isStatefulPod=false podCreationTimestamp="2026-09-15 16:48:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:48:50.377974865 +0000 UTC m=+1347.007629217" watchObservedRunningTime="2026-09-15 16:48:50.37985406 +0000 UTC m=+1347.009508449"
Sep 15 16:48:51.362941 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:51.362909    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj" podUID="d701f09c-42b4-47d8-8ee4-23ebcd42dd54" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Sep 15 16:48:56.367630 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:56.367603    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj"
Sep 15 16:48:56.368056 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:48:56.368033    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj" podUID="d701f09c-42b4-47d8-8ee4-23ebcd42dd54" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Sep 15 16:49:06.368058 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:49:06.368019    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj" podUID="d701f09c-42b4-47d8-8ee4-23ebcd42dd54" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Sep 15 16:49:16.368558 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:49:16.368520    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj" podUID="d701f09c-42b4-47d8-8ee4-23ebcd42dd54" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Sep 15 16:49:26.368561 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:49:26.368529    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj" podUID="d701f09c-42b4-47d8-8ee4-23ebcd42dd54" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Sep 15 16:49:36.368567 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:49:36.368528    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj" podUID="d701f09c-42b4-47d8-8ee4-23ebcd42dd54" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Sep 15 16:49:46.368140 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:49:46.368094    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj" podUID="d701f09c-42b4-47d8-8ee4-23ebcd42dd54" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Sep 15 16:49:56.368959 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:49:56.368927    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj"
Sep 15 16:50:09.548013 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:09.547976    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj"]
Sep 15 16:50:09.548552 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:09.548230    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj" podUID="d701f09c-42b4-47d8-8ee4-23ebcd42dd54" containerName="kserve-container" containerID="cri-o://3fbad23d92166cf78f63d32f6c1cb57a4310b8d2513f5a35b5d7dafc5affd5bc" gracePeriod=30
Sep 15 16:50:09.548552 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:09.548259    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj" podUID="d701f09c-42b4-47d8-8ee4-23ebcd42dd54" containerName="kube-rbac-proxy" containerID="cri-o://78d45c05f310877abb2fe8ce17aa0fec53a98bb80b18d6c9cbea6a46a1538c7d" gracePeriod=30
Sep 15 16:50:09.713858 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:09.713821    2579 generic.go:356] "Generic (PLEG): container finished" podID="d701f09c-42b4-47d8-8ee4-23ebcd42dd54" containerID="78d45c05f310877abb2fe8ce17aa0fec53a98bb80b18d6c9cbea6a46a1538c7d" exitCode=2
Sep 15 16:50:09.714020 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:09.713893    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj" event={"ID":"d701f09c-42b4-47d8-8ee4-23ebcd42dd54","Type":"ContainerDied","Data":"78d45c05f310877abb2fe8ce17aa0fec53a98bb80b18d6c9cbea6a46a1538c7d"}
Sep 15 16:50:11.363358 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:11.363310    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj" podUID="d701f09c-42b4-47d8-8ee4-23ebcd42dd54" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.34:8643/healthz\": dial tcp 10.133.0.34:8643: connect: connection refused"
Sep 15 16:50:14.392066 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.392039    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj"
Sep 15 16:50:14.527759 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.527674    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d701f09c-42b4-47d8-8ee4-23ebcd42dd54-proxy-tls\") pod \"d701f09c-42b4-47d8-8ee4-23ebcd42dd54\" (UID: \"d701f09c-42b4-47d8-8ee4-23ebcd42dd54\") "
Sep 15 16:50:14.527759 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.527746    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d701f09c-42b4-47d8-8ee4-23ebcd42dd54-kube-rbac-proxy-sar-config\") pod \"d701f09c-42b4-47d8-8ee4-23ebcd42dd54\" (UID: \"d701f09c-42b4-47d8-8ee4-23ebcd42dd54\") "
Sep 15 16:50:14.528002 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.527812    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d701f09c-42b4-47d8-8ee4-23ebcd42dd54-kserve-provision-location\") pod \"d701f09c-42b4-47d8-8ee4-23ebcd42dd54\" (UID: \"d701f09c-42b4-47d8-8ee4-23ebcd42dd54\") "
Sep 15 16:50:14.528002 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.527848    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d701f09c-42b4-47d8-8ee4-23ebcd42dd54-cabundle-cert\") pod \"d701f09c-42b4-47d8-8ee4-23ebcd42dd54\" (UID: \"d701f09c-42b4-47d8-8ee4-23ebcd42dd54\") "
Sep 15 16:50:14.528002 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.527881    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dvwz\" (UniqueName: \"kubernetes.io/projected/d701f09c-42b4-47d8-8ee4-23ebcd42dd54-kube-api-access-7dvwz\") pod \"d701f09c-42b4-47d8-8ee4-23ebcd42dd54\" (UID: \"d701f09c-42b4-47d8-8ee4-23ebcd42dd54\") "
Sep 15 16:50:14.528220 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.528183    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d701f09c-42b4-47d8-8ee4-23ebcd42dd54-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "d701f09c-42b4-47d8-8ee4-23ebcd42dd54" (UID: "d701f09c-42b4-47d8-8ee4-23ebcd42dd54"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 16:50:14.528220 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.528198    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d701f09c-42b4-47d8-8ee4-23ebcd42dd54-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "d701f09c-42b4-47d8-8ee4-23ebcd42dd54" (UID: "d701f09c-42b4-47d8-8ee4-23ebcd42dd54"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:50:14.528220 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.528207    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d701f09c-42b4-47d8-8ee4-23ebcd42dd54-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "d701f09c-42b4-47d8-8ee4-23ebcd42dd54" (UID: "d701f09c-42b4-47d8-8ee4-23ebcd42dd54"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:50:14.530016 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.529992    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d701f09c-42b4-47d8-8ee4-23ebcd42dd54-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d701f09c-42b4-47d8-8ee4-23ebcd42dd54" (UID: "d701f09c-42b4-47d8-8ee4-23ebcd42dd54"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 16:50:14.530102 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.530033    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d701f09c-42b4-47d8-8ee4-23ebcd42dd54-kube-api-access-7dvwz" (OuterVolumeSpecName: "kube-api-access-7dvwz") pod "d701f09c-42b4-47d8-8ee4-23ebcd42dd54" (UID: "d701f09c-42b4-47d8-8ee4-23ebcd42dd54"). InnerVolumeSpecName "kube-api-access-7dvwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 16:50:14.629369 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.629331    2579 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d701f09c-42b4-47d8-8ee4-23ebcd42dd54-cabundle-cert\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:50:14.629369 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.629362    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dvwz\" (UniqueName: \"kubernetes.io/projected/d701f09c-42b4-47d8-8ee4-23ebcd42dd54-kube-api-access-7dvwz\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:50:14.629369 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.629374    2579 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d701f09c-42b4-47d8-8ee4-23ebcd42dd54-proxy-tls\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:50:14.629600 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.629384    2579 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d701f09c-42b4-47d8-8ee4-23ebcd42dd54-kube-rbac-proxy-sar-config\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:50:14.629600 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.629392    2579 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d701f09c-42b4-47d8-8ee4-23ebcd42dd54-kserve-provision-location\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:50:14.737578 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.737546    2579 generic.go:356] "Generic (PLEG): container finished" podID="d701f09c-42b4-47d8-8ee4-23ebcd42dd54" containerID="3fbad23d92166cf78f63d32f6c1cb57a4310b8d2513f5a35b5d7dafc5affd5bc" exitCode=0
Sep 15 16:50:14.737761 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.737614    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj"
Sep 15 16:50:14.737761 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.737633    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj" event={"ID":"d701f09c-42b4-47d8-8ee4-23ebcd42dd54","Type":"ContainerDied","Data":"3fbad23d92166cf78f63d32f6c1cb57a4310b8d2513f5a35b5d7dafc5affd5bc"}
Sep 15 16:50:14.737761 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.737698    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj" event={"ID":"d701f09c-42b4-47d8-8ee4-23ebcd42dd54","Type":"ContainerDied","Data":"b58d47a19e666da532ce3524ab8fbf430dbb251034c07935279f7cdddbb8fe2d"}
Sep 15 16:50:14.737761 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.737727    2579 scope.go:117] "RemoveContainer" containerID="78d45c05f310877abb2fe8ce17aa0fec53a98bb80b18d6c9cbea6a46a1538c7d"
Sep 15 16:50:14.747528 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.747500    2579 scope.go:117] "RemoveContainer" containerID="3fbad23d92166cf78f63d32f6c1cb57a4310b8d2513f5a35b5d7dafc5affd5bc"
Sep 15 16:50:14.754439 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.754414    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v"]
Sep 15 16:50:14.754899 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.754875    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01c8e8e1-1248-4175-bb6d-2dd059baef67" containerName="kube-rbac-proxy"
Sep 15 16:50:14.754899 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.754900    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="01c8e8e1-1248-4175-bb6d-2dd059baef67" containerName="kube-rbac-proxy"
Sep 15 16:50:14.755132 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.754912    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d701f09c-42b4-47d8-8ee4-23ebcd42dd54" containerName="kserve-container"
Sep 15 16:50:14.755132 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.754920    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d701f09c-42b4-47d8-8ee4-23ebcd42dd54" containerName="kserve-container"
Sep 15 16:50:14.755132 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.754930    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d701f09c-42b4-47d8-8ee4-23ebcd42dd54" containerName="kube-rbac-proxy"
Sep 15 16:50:14.755132 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.754938    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d701f09c-42b4-47d8-8ee4-23ebcd42dd54" containerName="kube-rbac-proxy"
Sep 15 16:50:14.755132 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.754965    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01c8e8e1-1248-4175-bb6d-2dd059baef67" containerName="storage-initializer"
Sep 15 16:50:14.755132 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.754973    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="01c8e8e1-1248-4175-bb6d-2dd059baef67" containerName="storage-initializer"
Sep 15 16:50:14.755132 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.754988    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01c8e8e1-1248-4175-bb6d-2dd059baef67" containerName="kserve-container"
Sep 15 16:50:14.755132 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.754995    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="01c8e8e1-1248-4175-bb6d-2dd059baef67" containerName="kserve-container"
Sep 15 16:50:14.755132 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.755004    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d701f09c-42b4-47d8-8ee4-23ebcd42dd54" containerName="storage-initializer"
Sep 15 16:50:14.755132 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.755011    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d701f09c-42b4-47d8-8ee4-23ebcd42dd54" containerName="storage-initializer"
Sep 15 16:50:14.755132 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.755099    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d701f09c-42b4-47d8-8ee4-23ebcd42dd54" containerName="kserve-container"
Sep 15 16:50:14.755132 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.755113    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="01c8e8e1-1248-4175-bb6d-2dd059baef67" containerName="kube-rbac-proxy"
Sep 15 16:50:14.755132 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.755123    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d701f09c-42b4-47d8-8ee4-23ebcd42dd54" containerName="kube-rbac-proxy"
Sep 15 16:50:14.755132 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.755132    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="01c8e8e1-1248-4175-bb6d-2dd059baef67" containerName="kserve-container"
Sep 15 16:50:14.758148 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.758128    2579 scope.go:117] "RemoveContainer" containerID="59a18fd5805445464cec4b04e9399078e7ab5c77e007b33e1d909f1de9c69a95"
Sep 15 16:50:14.760227 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.760207    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v"
Sep 15 16:50:14.763131 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.763110    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-promote-predictor-serving-cert\""
Sep 15 16:50:14.763776 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.763751    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 15 16:50:14.764108 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.764092    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Sep 15 16:50:14.764275 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.764256    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-lqw4s\""
Sep 15 16:50:14.764487 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.764200    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 15 16:50:14.764604 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.764110    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Sep 15 16:50:14.766090 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.765940    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-promote-kube-rbac-proxy-sar-config\""
Sep 15 16:50:14.767043 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.766994    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj"]
Sep 15 16:50:14.769532 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.769509    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v"]
Sep 15 16:50:14.771608 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.771585    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-create-predictor-7d5b67cc88-gzdsj"]
Sep 15 16:50:14.773241 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.773223    2579 scope.go:117] "RemoveContainer" containerID="78d45c05f310877abb2fe8ce17aa0fec53a98bb80b18d6c9cbea6a46a1538c7d"
Sep 15 16:50:14.773581 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:50:14.773558    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78d45c05f310877abb2fe8ce17aa0fec53a98bb80b18d6c9cbea6a46a1538c7d\": container with ID starting with 78d45c05f310877abb2fe8ce17aa0fec53a98bb80b18d6c9cbea6a46a1538c7d not found: ID does not exist" containerID="78d45c05f310877abb2fe8ce17aa0fec53a98bb80b18d6c9cbea6a46a1538c7d"
Sep 15 16:50:14.773685 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.773594    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78d45c05f310877abb2fe8ce17aa0fec53a98bb80b18d6c9cbea6a46a1538c7d"} err="failed to get container status \"78d45c05f310877abb2fe8ce17aa0fec53a98bb80b18d6c9cbea6a46a1538c7d\": rpc error: code = NotFound desc = could not find container \"78d45c05f310877abb2fe8ce17aa0fec53a98bb80b18d6c9cbea6a46a1538c7d\": container with ID starting with 78d45c05f310877abb2fe8ce17aa0fec53a98bb80b18d6c9cbea6a46a1538c7d not found: ID does not exist"
Sep 15 16:50:14.773685 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.773618    2579 scope.go:117] "RemoveContainer" containerID="3fbad23d92166cf78f63d32f6c1cb57a4310b8d2513f5a35b5d7dafc5affd5bc"
Sep 15 16:50:14.774136 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:50:14.774112    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fbad23d92166cf78f63d32f6c1cb57a4310b8d2513f5a35b5d7dafc5affd5bc\": container with ID starting with 3fbad23d92166cf78f63d32f6c1cb57a4310b8d2513f5a35b5d7dafc5affd5bc not found: ID does not exist" containerID="3fbad23d92166cf78f63d32f6c1cb57a4310b8d2513f5a35b5d7dafc5affd5bc"
Sep 15 16:50:14.774232 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.774143    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fbad23d92166cf78f63d32f6c1cb57a4310b8d2513f5a35b5d7dafc5affd5bc"} err="failed to get container status \"3fbad23d92166cf78f63d32f6c1cb57a4310b8d2513f5a35b5d7dafc5affd5bc\": rpc error: code = NotFound desc = could not find container \"3fbad23d92166cf78f63d32f6c1cb57a4310b8d2513f5a35b5d7dafc5affd5bc\": container with ID starting with 3fbad23d92166cf78f63d32f6c1cb57a4310b8d2513f5a35b5d7dafc5affd5bc not found: ID does not exist"
Sep 15 16:50:14.774232 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.774163    2579 scope.go:117] "RemoveContainer" containerID="59a18fd5805445464cec4b04e9399078e7ab5c77e007b33e1d909f1de9c69a95"
Sep 15 16:50:14.774552 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:50:14.774516    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59a18fd5805445464cec4b04e9399078e7ab5c77e007b33e1d909f1de9c69a95\": container with ID starting with 59a18fd5805445464cec4b04e9399078e7ab5c77e007b33e1d909f1de9c69a95 not found: ID does not exist" containerID="59a18fd5805445464cec4b04e9399078e7ab5c77e007b33e1d909f1de9c69a95"
Sep 15 16:50:14.775592 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.774553    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59a18fd5805445464cec4b04e9399078e7ab5c77e007b33e1d909f1de9c69a95"} err="failed to get container status \"59a18fd5805445464cec4b04e9399078e7ab5c77e007b33e1d909f1de9c69a95\": rpc error: code = NotFound desc = could not find container \"59a18fd5805445464cec4b04e9399078e7ab5c77e007b33e1d909f1de9c69a95\": container with ID starting with 59a18fd5805445464cec4b04e9399078e7ab5c77e007b33e1d909f1de9c69a95 not found: ID does not exist"
Sep 15 16:50:14.831395 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.831308    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-proxy-tls\") pod \"isvc-canary-promote-predictor-6d6bc676fb-gsh4v\" (UID: \"7c27f8f0-b46d-419c-8af9-3e81ac545e9d\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v"
Sep 15 16:50:14.831395 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.831354    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-cabundle-cert\") pod \"isvc-canary-promote-predictor-6d6bc676fb-gsh4v\" (UID: \"7c27f8f0-b46d-419c-8af9-3e81ac545e9d\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v"
Sep 15 16:50:14.831581 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.831439    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-kserve-provision-location\") pod \"isvc-canary-promote-predictor-6d6bc676fb-gsh4v\" (UID: \"7c27f8f0-b46d-419c-8af9-3e81ac545e9d\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v"
Sep 15 16:50:14.831581 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.831485    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-6d6bc676fb-gsh4v\" (UID: \"7c27f8f0-b46d-419c-8af9-3e81ac545e9d\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v"
Sep 15 16:50:14.831581 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.831513    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8q6r\" (UniqueName: \"kubernetes.io/projected/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-kube-api-access-h8q6r\") pod \"isvc-canary-promote-predictor-6d6bc676fb-gsh4v\" (UID: \"7c27f8f0-b46d-419c-8af9-3e81ac545e9d\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v"
Sep 15 16:50:14.932555 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.932510    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-kserve-provision-location\") pod \"isvc-canary-promote-predictor-6d6bc676fb-gsh4v\" (UID: \"7c27f8f0-b46d-419c-8af9-3e81ac545e9d\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v"
Sep 15 16:50:14.932750 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.932563    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-6d6bc676fb-gsh4v\" (UID: \"7c27f8f0-b46d-419c-8af9-3e81ac545e9d\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v"
Sep 15 16:50:14.932750 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.932582    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8q6r\" (UniqueName: \"kubernetes.io/projected/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-kube-api-access-h8q6r\") pod \"isvc-canary-promote-predictor-6d6bc676fb-gsh4v\" (UID: \"7c27f8f0-b46d-419c-8af9-3e81ac545e9d\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v"
Sep 15 16:50:14.932750 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.932695    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-proxy-tls\") pod \"isvc-canary-promote-predictor-6d6bc676fb-gsh4v\" (UID: \"7c27f8f0-b46d-419c-8af9-3e81ac545e9d\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v"
Sep 15 16:50:14.932750 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.932740    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-cabundle-cert\") pod \"isvc-canary-promote-predictor-6d6bc676fb-gsh4v\" (UID: \"7c27f8f0-b46d-419c-8af9-3e81ac545e9d\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v"
Sep 15 16:50:14.932995 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:50:14.932936    2579 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-canary-promote-predictor-serving-cert: secret "isvc-canary-promote-predictor-serving-cert" not found
Sep 15 16:50:14.933053 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:50:14.933012    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-proxy-tls podName:7c27f8f0-b46d-419c-8af9-3e81ac545e9d nodeName:}" failed. No retries permitted until 2026-09-15 16:50:15.432989526 +0000 UTC m=+1432.062643857 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-proxy-tls") pod "isvc-canary-promote-predictor-6d6bc676fb-gsh4v" (UID: "7c27f8f0-b46d-419c-8af9-3e81ac545e9d") : secret "isvc-canary-promote-predictor-serving-cert" not found
Sep 15 16:50:14.933445 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.933424    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-6d6bc676fb-gsh4v\" (UID: \"7c27f8f0-b46d-419c-8af9-3e81ac545e9d\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v"
Sep 15 16:50:14.933500 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.933426    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-kserve-provision-location\") pod \"isvc-canary-promote-predictor-6d6bc676fb-gsh4v\" (UID: \"7c27f8f0-b46d-419c-8af9-3e81ac545e9d\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v"
Sep 15 16:50:14.933500 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.933478    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-cabundle-cert\") pod \"isvc-canary-promote-predictor-6d6bc676fb-gsh4v\" (UID: \"7c27f8f0-b46d-419c-8af9-3e81ac545e9d\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v"
Sep 15 16:50:14.941016 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:14.940997    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8q6r\" (UniqueName: \"kubernetes.io/projected/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-kube-api-access-h8q6r\") pod \"isvc-canary-promote-predictor-6d6bc676fb-gsh4v\" (UID: \"7c27f8f0-b46d-419c-8af9-3e81ac545e9d\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v"
Sep 15 16:50:15.436836 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:15.436770    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-proxy-tls\") pod \"isvc-canary-promote-predictor-6d6bc676fb-gsh4v\" (UID: \"7c27f8f0-b46d-419c-8af9-3e81ac545e9d\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v"
Sep 15 16:50:15.439342 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:15.439313    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-proxy-tls\") pod \"isvc-canary-promote-predictor-6d6bc676fb-gsh4v\" (UID: \"7c27f8f0-b46d-419c-8af9-3e81ac545e9d\") " pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v"
Sep 15 16:50:15.676750 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:15.676705    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v"
Sep 15 16:50:15.909092 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:15.909059    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d701f09c-42b4-47d8-8ee4-23ebcd42dd54" path="/var/lib/kubelet/pods/d701f09c-42b4-47d8-8ee4-23ebcd42dd54/volumes"
Sep 15 16:50:15.951079 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:15.951054    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v"]
Sep 15 16:50:15.953234 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:50:15.953209    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c27f8f0_b46d_419c_8af9_3e81ac545e9d.slice/crio-dd453e224da66db28e763319486c8e71d22e64287fb6febb348b49aacd609187 WatchSource:0}: Error finding container dd453e224da66db28e763319486c8e71d22e64287fb6febb348b49aacd609187: Status 404 returned error can't find the container with id dd453e224da66db28e763319486c8e71d22e64287fb6febb348b49aacd609187
Sep 15 16:50:16.750047 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:16.750005    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v" event={"ID":"7c27f8f0-b46d-419c-8af9-3e81ac545e9d","Type":"ContainerStarted","Data":"73502cfa5bf76ad64379bfd9f84f51ce00f8ff35d979c29bb63d7356417104bb"}
Sep 15 16:50:16.750414 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:16.750111    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v" event={"ID":"7c27f8f0-b46d-419c-8af9-3e81ac545e9d","Type":"ContainerStarted","Data":"dd453e224da66db28e763319486c8e71d22e64287fb6febb348b49aacd609187"}
Sep 15 16:50:20.768729 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:20.768692    2579 generic.go:356] "Generic (PLEG): container finished" podID="7c27f8f0-b46d-419c-8af9-3e81ac545e9d" containerID="73502cfa5bf76ad64379bfd9f84f51ce00f8ff35d979c29bb63d7356417104bb" exitCode=0
Sep 15 16:50:20.769202 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:20.768756    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v" event={"ID":"7c27f8f0-b46d-419c-8af9-3e81ac545e9d","Type":"ContainerDied","Data":"73502cfa5bf76ad64379bfd9f84f51ce00f8ff35d979c29bb63d7356417104bb"}
Sep 15 16:50:21.775897 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:21.775862    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v" event={"ID":"7c27f8f0-b46d-419c-8af9-3e81ac545e9d","Type":"ContainerStarted","Data":"ddd9289bf80c6f388c0e0d5525d447d0c6a699a5f90b631f12a1eb0577663ef7"}
Sep 15 16:50:21.776328 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:21.775924    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v" event={"ID":"7c27f8f0-b46d-419c-8af9-3e81ac545e9d","Type":"ContainerStarted","Data":"1c131ee36f52ddb66900dc157344dfebca3c321a85b0da1b6552c592b5119b59"}
Sep 15 16:50:21.776328 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:21.776191    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v"
Sep 15 16:50:21.776454 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:21.776330    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v"
Sep 15 16:50:21.777571 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:21.777541    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v" podUID="7c27f8f0-b46d-419c-8af9-3e81ac545e9d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 15 16:50:21.794665 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:21.794616    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v" podStartSLOduration=4.043050082 podStartE2EDuration="7.794599077s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.751548995s" isStatefulPod=false podCreationTimestamp="2026-09-15 16:50:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:50:21.793399418 +0000 UTC m=+1438.423053769" watchObservedRunningTime="2026-09-15 16:50:21.794599077 +0000 UTC m=+1438.424253428"
Sep 15 16:50:22.781295 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:22.781254    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v" podUID="7c27f8f0-b46d-419c-8af9-3e81ac545e9d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 15 16:50:27.785803 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:27.785750    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v"
Sep 15 16:50:27.786218 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:27.786188    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v" podUID="7c27f8f0-b46d-419c-8af9-3e81ac545e9d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 15 16:50:37.786635 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:37.786596    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v" podUID="7c27f8f0-b46d-419c-8af9-3e81ac545e9d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 15 16:50:47.787207 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:47.787164    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v" podUID="7c27f8f0-b46d-419c-8af9-3e81ac545e9d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 15 16:50:57.787012 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:50:57.786964    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v" podUID="7c27f8f0-b46d-419c-8af9-3e81ac545e9d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 15 16:51:07.786220 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:07.786178    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v" podUID="7c27f8f0-b46d-419c-8af9-3e81ac545e9d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 15 16:51:17.787077 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:17.787039    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v" podUID="7c27f8f0-b46d-419c-8af9-3e81ac545e9d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 15 16:51:23.920583 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:23.920552    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4rdn_44ca88f0-c149-443e-a2d1-a30964775818/ovn-acl-logging/0.log"
Sep 15 16:51:23.923617 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:23.923590    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4rdn_44ca88f0-c149-443e-a2d1-a30964775818/ovn-acl-logging/0.log"
Sep 15 16:51:27.786946 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:27.786905    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v" podUID="7c27f8f0-b46d-419c-8af9-3e81ac545e9d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 15 16:51:34.904847 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:34.904805    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v"
Sep 15 16:51:44.804242 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:44.804209    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v"]
Sep 15 16:51:44.804727 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:44.804682    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v" podUID="7c27f8f0-b46d-419c-8af9-3e81ac545e9d" containerName="kserve-container" containerID="cri-o://1c131ee36f52ddb66900dc157344dfebca3c321a85b0da1b6552c592b5119b59" gracePeriod=30
Sep 15 16:51:44.804849 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:44.804711    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v" podUID="7c27f8f0-b46d-419c-8af9-3e81ac545e9d" containerName="kube-rbac-proxy" containerID="cri-o://ddd9289bf80c6f388c0e0d5525d447d0c6a699a5f90b631f12a1eb0577663ef7" gracePeriod=30
Sep 15 16:51:44.903995 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:44.903955    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v" podUID="7c27f8f0-b46d-419c-8af9-3e81ac545e9d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 15 16:51:44.922117 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:44.922077    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv"]
Sep 15 16:51:44.925494 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:44.925472    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv"
Sep 15 16:51:44.928214 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:44.928187    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-promote-v2-predictor-serving-cert\""
Sep 15 16:51:44.935580 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:44.935558    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv"]
Sep 15 16:51:45.003268 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:45.003233    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8e9e377b-388d-4e0a-b41f-a41b96324ea2-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv\" (UID: \"8e9e377b-388d-4e0a-b41f-a41b96324ea2\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv"
Sep 15 16:51:45.003268 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:45.003270    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8e9e377b-388d-4e0a-b41f-a41b96324ea2-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv\" (UID: \"8e9e377b-388d-4e0a-b41f-a41b96324ea2\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv"
Sep 15 16:51:45.003527 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:45.003298    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lb7rn\" (UniqueName: \"kubernetes.io/projected/8e9e377b-388d-4e0a-b41f-a41b96324ea2-kube-api-access-lb7rn\") pod \"isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv\" (UID: \"8e9e377b-388d-4e0a-b41f-a41b96324ea2\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv"
Sep 15 16:51:45.003527 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:45.003346    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8e9e377b-388d-4e0a-b41f-a41b96324ea2-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv\" (UID: \"8e9e377b-388d-4e0a-b41f-a41b96324ea2\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv"
Sep 15 16:51:45.003527 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:45.003384    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8e9e377b-388d-4e0a-b41f-a41b96324ea2-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv\" (UID: \"8e9e377b-388d-4e0a-b41f-a41b96324ea2\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv"
Sep 15 16:51:45.104313 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:45.104185    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb7rn\" (UniqueName: \"kubernetes.io/projected/8e9e377b-388d-4e0a-b41f-a41b96324ea2-kube-api-access-lb7rn\") pod \"isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv\" (UID: \"8e9e377b-388d-4e0a-b41f-a41b96324ea2\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv"
Sep 15 16:51:45.104313 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:45.104219    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8e9e377b-388d-4e0a-b41f-a41b96324ea2-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv\" (UID: \"8e9e377b-388d-4e0a-b41f-a41b96324ea2\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv"
Sep 15 16:51:45.104313 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:45.104254    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8e9e377b-388d-4e0a-b41f-a41b96324ea2-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv\" (UID: \"8e9e377b-388d-4e0a-b41f-a41b96324ea2\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv"
Sep 15 16:51:45.104591 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:45.104337    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8e9e377b-388d-4e0a-b41f-a41b96324ea2-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv\" (UID: \"8e9e377b-388d-4e0a-b41f-a41b96324ea2\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv"
Sep 15 16:51:45.104591 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:45.104368    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8e9e377b-388d-4e0a-b41f-a41b96324ea2-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv\" (UID: \"8e9e377b-388d-4e0a-b41f-a41b96324ea2\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv"
Sep 15 16:51:45.104745 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:45.104722    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8e9e377b-388d-4e0a-b41f-a41b96324ea2-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv\" (UID: \"8e9e377b-388d-4e0a-b41f-a41b96324ea2\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv"
Sep 15 16:51:45.105066 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:45.105041    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8e9e377b-388d-4e0a-b41f-a41b96324ea2-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv\" (UID: \"8e9e377b-388d-4e0a-b41f-a41b96324ea2\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv"
Sep 15 16:51:45.105172 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:45.105041    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8e9e377b-388d-4e0a-b41f-a41b96324ea2-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv\" (UID: \"8e9e377b-388d-4e0a-b41f-a41b96324ea2\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv"
Sep 15 16:51:45.107033 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:45.107006    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8e9e377b-388d-4e0a-b41f-a41b96324ea2-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv\" (UID: \"8e9e377b-388d-4e0a-b41f-a41b96324ea2\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv"
Sep 15 16:51:45.112497 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:45.112475    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb7rn\" (UniqueName: \"kubernetes.io/projected/8e9e377b-388d-4e0a-b41f-a41b96324ea2-kube-api-access-lb7rn\") pod \"isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv\" (UID: \"8e9e377b-388d-4e0a-b41f-a41b96324ea2\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv"
Sep 15 16:51:45.145654 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:45.145619    2579 generic.go:356] "Generic (PLEG): container finished" podID="7c27f8f0-b46d-419c-8af9-3e81ac545e9d" containerID="ddd9289bf80c6f388c0e0d5525d447d0c6a699a5f90b631f12a1eb0577663ef7" exitCode=2
Sep 15 16:51:45.145808 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:45.145694    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v" event={"ID":"7c27f8f0-b46d-419c-8af9-3e81ac545e9d","Type":"ContainerDied","Data":"ddd9289bf80c6f388c0e0d5525d447d0c6a699a5f90b631f12a1eb0577663ef7"}
Sep 15 16:51:45.236479 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:45.236445    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv"
Sep 15 16:51:45.513990 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:45.513905    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv"]
Sep 15 16:51:45.516201 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:51:45.516179    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e9e377b_388d_4e0a_b41f_a41b96324ea2.slice/crio-4f059a9ccd86d9b261de1b0ef5ea6dd5e9e4a4320219a161b2461cc565c4d145 WatchSource:0}: Error finding container 4f059a9ccd86d9b261de1b0ef5ea6dd5e9e4a4320219a161b2461cc565c4d145: Status 404 returned error can't find the container with id 4f059a9ccd86d9b261de1b0ef5ea6dd5e9e4a4320219a161b2461cc565c4d145
Sep 15 16:51:46.153176 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:46.153142    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv" event={"ID":"8e9e377b-388d-4e0a-b41f-a41b96324ea2","Type":"ContainerStarted","Data":"eb1d507ccee2a50282e8794b65976c56ee93f5448c2b6b9943f17aab64bd72cc"}
Sep 15 16:51:46.153593 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:46.153201    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv" event={"ID":"8e9e377b-388d-4e0a-b41f-a41b96324ea2","Type":"ContainerStarted","Data":"4f059a9ccd86d9b261de1b0ef5ea6dd5e9e4a4320219a161b2461cc565c4d145"}
Sep 15 16:51:47.782190 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:47.782141    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v" podUID="7c27f8f0-b46d-419c-8af9-3e81ac545e9d" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.35:8643/healthz\": dial tcp 10.133.0.35:8643: connect: connection refused"
Sep 15 16:51:49.661547 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:49.661521    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v"
Sep 15 16:51:49.743425 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:49.743395    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-proxy-tls\") pod \"7c27f8f0-b46d-419c-8af9-3e81ac545e9d\" (UID: \"7c27f8f0-b46d-419c-8af9-3e81ac545e9d\") "
Sep 15 16:51:49.743589 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:49.743435    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-cabundle-cert\") pod \"7c27f8f0-b46d-419c-8af9-3e81ac545e9d\" (UID: \"7c27f8f0-b46d-419c-8af9-3e81ac545e9d\") "
Sep 15 16:51:49.743589 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:49.743478    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8q6r\" (UniqueName: \"kubernetes.io/projected/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-kube-api-access-h8q6r\") pod \"7c27f8f0-b46d-419c-8af9-3e81ac545e9d\" (UID: \"7c27f8f0-b46d-419c-8af9-3e81ac545e9d\") "
Sep 15 16:51:49.743589 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:49.743519    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-kserve-provision-location\") pod \"7c27f8f0-b46d-419c-8af9-3e81ac545e9d\" (UID: \"7c27f8f0-b46d-419c-8af9-3e81ac545e9d\") "
Sep 15 16:51:49.743589 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:49.743546    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-kube-rbac-proxy-sar-config\") pod \"7c27f8f0-b46d-419c-8af9-3e81ac545e9d\" (UID: \"7c27f8f0-b46d-419c-8af9-3e81ac545e9d\") "
Sep 15 16:51:49.743851 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:49.743831    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "7c27f8f0-b46d-419c-8af9-3e81ac545e9d" (UID: "7c27f8f0-b46d-419c-8af9-3e81ac545e9d"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 16:51:49.743912 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:49.743888    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "7c27f8f0-b46d-419c-8af9-3e81ac545e9d" (UID: "7c27f8f0-b46d-419c-8af9-3e81ac545e9d"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:51:49.743962 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:49.743932    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "7c27f8f0-b46d-419c-8af9-3e81ac545e9d" (UID: "7c27f8f0-b46d-419c-8af9-3e81ac545e9d"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:51:49.745708 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:49.745687    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-kube-api-access-h8q6r" (OuterVolumeSpecName: "kube-api-access-h8q6r") pod "7c27f8f0-b46d-419c-8af9-3e81ac545e9d" (UID: "7c27f8f0-b46d-419c-8af9-3e81ac545e9d"). InnerVolumeSpecName "kube-api-access-h8q6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 16:51:49.745769 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:49.745706    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "7c27f8f0-b46d-419c-8af9-3e81ac545e9d" (UID: "7c27f8f0-b46d-419c-8af9-3e81ac545e9d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 16:51:49.845199 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:49.845111    2579 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-kserve-provision-location\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:51:49.845199 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:49.845144    2579 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-kube-rbac-proxy-sar-config\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:51:49.845199 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:49.845155    2579 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-proxy-tls\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:51:49.845199 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:49.845166    2579 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-cabundle-cert\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:51:49.845199 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:49.845174    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8q6r\" (UniqueName: \"kubernetes.io/projected/7c27f8f0-b46d-419c-8af9-3e81ac545e9d-kube-api-access-h8q6r\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:51:50.171277 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:50.171181    2579 generic.go:356] "Generic (PLEG): container finished" podID="8e9e377b-388d-4e0a-b41f-a41b96324ea2" containerID="eb1d507ccee2a50282e8794b65976c56ee93f5448c2b6b9943f17aab64bd72cc" exitCode=0
Sep 15 16:51:50.171277 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:50.171256    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv" event={"ID":"8e9e377b-388d-4e0a-b41f-a41b96324ea2","Type":"ContainerDied","Data":"eb1d507ccee2a50282e8794b65976c56ee93f5448c2b6b9943f17aab64bd72cc"}
Sep 15 16:51:50.173298 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:50.173276    2579 generic.go:356] "Generic (PLEG): container finished" podID="7c27f8f0-b46d-419c-8af9-3e81ac545e9d" containerID="1c131ee36f52ddb66900dc157344dfebca3c321a85b0da1b6552c592b5119b59" exitCode=0
Sep 15 16:51:50.173407 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:50.173334    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v"
Sep 15 16:51:50.173407 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:50.173350    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v" event={"ID":"7c27f8f0-b46d-419c-8af9-3e81ac545e9d","Type":"ContainerDied","Data":"1c131ee36f52ddb66900dc157344dfebca3c321a85b0da1b6552c592b5119b59"}
Sep 15 16:51:50.173477 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:50.173409    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v" event={"ID":"7c27f8f0-b46d-419c-8af9-3e81ac545e9d","Type":"ContainerDied","Data":"dd453e224da66db28e763319486c8e71d22e64287fb6febb348b49aacd609187"}
Sep 15 16:51:50.173477 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:50.173438    2579 scope.go:117] "RemoveContainer" containerID="ddd9289bf80c6f388c0e0d5525d447d0c6a699a5f90b631f12a1eb0577663ef7"
Sep 15 16:51:50.183802 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:50.183760    2579 scope.go:117] "RemoveContainer" containerID="1c131ee36f52ddb66900dc157344dfebca3c321a85b0da1b6552c592b5119b59"
Sep 15 16:51:50.192678 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:50.192657    2579 scope.go:117] "RemoveContainer" containerID="73502cfa5bf76ad64379bfd9f84f51ce00f8ff35d979c29bb63d7356417104bb"
Sep 15 16:51:50.201567 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:50.201542    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v"]
Sep 15 16:51:50.205930 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:50.205907    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-predictor-6d6bc676fb-gsh4v"]
Sep 15 16:51:50.207356 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:50.207339    2579 scope.go:117] "RemoveContainer" containerID="ddd9289bf80c6f388c0e0d5525d447d0c6a699a5f90b631f12a1eb0577663ef7"
Sep 15 16:51:50.207730 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:51:50.207697    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddd9289bf80c6f388c0e0d5525d447d0c6a699a5f90b631f12a1eb0577663ef7\": container with ID starting with ddd9289bf80c6f388c0e0d5525d447d0c6a699a5f90b631f12a1eb0577663ef7 not found: ID does not exist" containerID="ddd9289bf80c6f388c0e0d5525d447d0c6a699a5f90b631f12a1eb0577663ef7"
Sep 15 16:51:50.207855 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:50.207738    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddd9289bf80c6f388c0e0d5525d447d0c6a699a5f90b631f12a1eb0577663ef7"} err="failed to get container status \"ddd9289bf80c6f388c0e0d5525d447d0c6a699a5f90b631f12a1eb0577663ef7\": rpc error: code = NotFound desc = could not find container \"ddd9289bf80c6f388c0e0d5525d447d0c6a699a5f90b631f12a1eb0577663ef7\": container with ID starting with ddd9289bf80c6f388c0e0d5525d447d0c6a699a5f90b631f12a1eb0577663ef7 not found: ID does not exist"
Sep 15 16:51:50.207855 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:50.207762    2579 scope.go:117] "RemoveContainer" containerID="1c131ee36f52ddb66900dc157344dfebca3c321a85b0da1b6552c592b5119b59"
Sep 15 16:51:50.208116 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:51:50.208091    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c131ee36f52ddb66900dc157344dfebca3c321a85b0da1b6552c592b5119b59\": container with ID starting with 1c131ee36f52ddb66900dc157344dfebca3c321a85b0da1b6552c592b5119b59 not found: ID does not exist" containerID="1c131ee36f52ddb66900dc157344dfebca3c321a85b0da1b6552c592b5119b59"
Sep 15 16:51:50.208195 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:50.208128    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c131ee36f52ddb66900dc157344dfebca3c321a85b0da1b6552c592b5119b59"} err="failed to get container status \"1c131ee36f52ddb66900dc157344dfebca3c321a85b0da1b6552c592b5119b59\": rpc error: code = NotFound desc = could not find container \"1c131ee36f52ddb66900dc157344dfebca3c321a85b0da1b6552c592b5119b59\": container with ID starting with 1c131ee36f52ddb66900dc157344dfebca3c321a85b0da1b6552c592b5119b59 not found: ID does not exist"
Sep 15 16:51:50.208195 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:50.208149    2579 scope.go:117] "RemoveContainer" containerID="73502cfa5bf76ad64379bfd9f84f51ce00f8ff35d979c29bb63d7356417104bb"
Sep 15 16:51:50.208577 ip-10-0-141-35 kubenswrapper[2579]: E0915 16:51:50.208554    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73502cfa5bf76ad64379bfd9f84f51ce00f8ff35d979c29bb63d7356417104bb\": container with ID starting with 73502cfa5bf76ad64379bfd9f84f51ce00f8ff35d979c29bb63d7356417104bb not found: ID does not exist" containerID="73502cfa5bf76ad64379bfd9f84f51ce00f8ff35d979c29bb63d7356417104bb"
Sep 15 16:51:50.208738 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:50.208712    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73502cfa5bf76ad64379bfd9f84f51ce00f8ff35d979c29bb63d7356417104bb"} err="failed to get container status \"73502cfa5bf76ad64379bfd9f84f51ce00f8ff35d979c29bb63d7356417104bb\": rpc error: code = NotFound desc = could not find container \"73502cfa5bf76ad64379bfd9f84f51ce00f8ff35d979c29bb63d7356417104bb\": container with ID starting with 73502cfa5bf76ad64379bfd9f84f51ce00f8ff35d979c29bb63d7356417104bb not found: ID does not exist"
Sep 15 16:51:51.180739 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:51.180708    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv" event={"ID":"8e9e377b-388d-4e0a-b41f-a41b96324ea2","Type":"ContainerStarted","Data":"24ecc6a7d88670e3d9772dc021f042c0525ea19c4d58740325ebb48f135bdd64"}
Sep 15 16:51:51.181182 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:51.180771    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv" event={"ID":"8e9e377b-388d-4e0a-b41f-a41b96324ea2","Type":"ContainerStarted","Data":"e2b01b4e1cbe17c7727401eba1eaf0733225b1cd29095458b16deba6addf8b72"}
Sep 15 16:51:51.181182 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:51.181082    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv"
Sep 15 16:51:51.181182 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:51.181145    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv"
Sep 15 16:51:51.182242 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:51.182215    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv" podUID="8e9e377b-388d-4e0a-b41f-a41b96324ea2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 15 16:51:51.199728 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:51.199686    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv" podStartSLOduration=3.4664550800000002 podStartE2EDuration="7.199672948s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.733217868s" isStatefulPod=false podCreationTimestamp="2026-09-15 16:51:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:51:51.197703596 +0000 UTC m=+1527.827357950" watchObservedRunningTime="2026-09-15 16:51:51.199672948 +0000 UTC m=+1527.829327299"
Sep 15 16:51:51.908729 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:51.908695    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c27f8f0-b46d-419c-8af9-3e81ac545e9d" path="/var/lib/kubelet/pods/7c27f8f0-b46d-419c-8af9-3e81ac545e9d/volumes"
Sep 15 16:51:52.186529 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:52.186431    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv" podUID="8e9e377b-388d-4e0a-b41f-a41b96324ea2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 15 16:51:57.191103 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:57.191073    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv"
Sep 15 16:51:57.191559 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:51:57.191529    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv" podUID="8e9e377b-388d-4e0a-b41f-a41b96324ea2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 15 16:52:07.192200 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:52:07.192156    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv" podUID="8e9e377b-388d-4e0a-b41f-a41b96324ea2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 15 16:52:17.191634 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:52:17.191595    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv" podUID="8e9e377b-388d-4e0a-b41f-a41b96324ea2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 15 16:52:27.192346 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:52:27.192305    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv" podUID="8e9e377b-388d-4e0a-b41f-a41b96324ea2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 15 16:52:37.191711 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:52:37.191670    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv" podUID="8e9e377b-388d-4e0a-b41f-a41b96324ea2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 15 16:52:47.191929 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:52:47.191888    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv" podUID="8e9e377b-388d-4e0a-b41f-a41b96324ea2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 15 16:52:57.192202 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:52:57.192168    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv" podUID="8e9e377b-388d-4e0a-b41f-a41b96324ea2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 15 16:53:07.192894 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:07.192862    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv"
Sep 15 16:53:15.055526 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:15.055494    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv"]
Sep 15 16:53:15.056027 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:15.055723    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv" podUID="8e9e377b-388d-4e0a-b41f-a41b96324ea2" containerName="kserve-container" containerID="cri-o://e2b01b4e1cbe17c7727401eba1eaf0733225b1cd29095458b16deba6addf8b72" gracePeriod=30
Sep 15 16:53:15.056027 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:15.055760    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv" podUID="8e9e377b-388d-4e0a-b41f-a41b96324ea2" containerName="kube-rbac-proxy" containerID="cri-o://24ecc6a7d88670e3d9772dc021f042c0525ea19c4d58740325ebb48f135bdd64" gracePeriod=30
Sep 15 16:53:15.561452 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:15.561416    2579 generic.go:356] "Generic (PLEG): container finished" podID="8e9e377b-388d-4e0a-b41f-a41b96324ea2" containerID="24ecc6a7d88670e3d9772dc021f042c0525ea19c4d58740325ebb48f135bdd64" exitCode=2
Sep 15 16:53:15.561617 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:15.561496    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv" event={"ID":"8e9e377b-388d-4e0a-b41f-a41b96324ea2","Type":"ContainerDied","Data":"24ecc6a7d88670e3d9772dc021f042c0525ea19c4d58740325ebb48f135bdd64"}
Sep 15 16:53:17.187334 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:17.187295    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv" podUID="8e9e377b-388d-4e0a-b41f-a41b96324ea2" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.36:8643/healthz\": dial tcp 10.133.0.36:8643: connect: connection refused"
Sep 15 16:53:17.191714 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:17.191690    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv" podUID="8e9e377b-388d-4e0a-b41f-a41b96324ea2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Sep 15 16:53:19.581729 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:19.581696    2579 generic.go:356] "Generic (PLEG): container finished" podID="8e9e377b-388d-4e0a-b41f-a41b96324ea2" containerID="e2b01b4e1cbe17c7727401eba1eaf0733225b1cd29095458b16deba6addf8b72" exitCode=0
Sep 15 16:53:19.582057 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:19.581793    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv" event={"ID":"8e9e377b-388d-4e0a-b41f-a41b96324ea2","Type":"ContainerDied","Data":"e2b01b4e1cbe17c7727401eba1eaf0733225b1cd29095458b16deba6addf8b72"}
Sep 15 16:53:19.705218 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:19.705195    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv"
Sep 15 16:53:19.804161 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:19.804102    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8e9e377b-388d-4e0a-b41f-a41b96324ea2-kube-rbac-proxy-sar-config\") pod \"8e9e377b-388d-4e0a-b41f-a41b96324ea2\" (UID: \"8e9e377b-388d-4e0a-b41f-a41b96324ea2\") "
Sep 15 16:53:19.804161 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:19.804157    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lb7rn\" (UniqueName: \"kubernetes.io/projected/8e9e377b-388d-4e0a-b41f-a41b96324ea2-kube-api-access-lb7rn\") pod \"8e9e377b-388d-4e0a-b41f-a41b96324ea2\" (UID: \"8e9e377b-388d-4e0a-b41f-a41b96324ea2\") "
Sep 15 16:53:19.804336 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:19.804183    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8e9e377b-388d-4e0a-b41f-a41b96324ea2-cabundle-cert\") pod \"8e9e377b-388d-4e0a-b41f-a41b96324ea2\" (UID: \"8e9e377b-388d-4e0a-b41f-a41b96324ea2\") "
Sep 15 16:53:19.804336 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:19.804207    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8e9e377b-388d-4e0a-b41f-a41b96324ea2-proxy-tls\") pod \"8e9e377b-388d-4e0a-b41f-a41b96324ea2\" (UID: \"8e9e377b-388d-4e0a-b41f-a41b96324ea2\") "
Sep 15 16:53:19.804336 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:19.804236    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8e9e377b-388d-4e0a-b41f-a41b96324ea2-kserve-provision-location\") pod \"8e9e377b-388d-4e0a-b41f-a41b96324ea2\" (UID: \"8e9e377b-388d-4e0a-b41f-a41b96324ea2\") "
Sep 15 16:53:19.804585 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:19.804551    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e9e377b-388d-4e0a-b41f-a41b96324ea2-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "8e9e377b-388d-4e0a-b41f-a41b96324ea2" (UID: "8e9e377b-388d-4e0a-b41f-a41b96324ea2"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:53:19.804585 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:19.804575    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e9e377b-388d-4e0a-b41f-a41b96324ea2-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "8e9e377b-388d-4e0a-b41f-a41b96324ea2" (UID: "8e9e377b-388d-4e0a-b41f-a41b96324ea2"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 16:53:19.804705 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:19.804602    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e9e377b-388d-4e0a-b41f-a41b96324ea2-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "8e9e377b-388d-4e0a-b41f-a41b96324ea2" (UID: "8e9e377b-388d-4e0a-b41f-a41b96324ea2"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 16:53:19.806474 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:19.806446    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e9e377b-388d-4e0a-b41f-a41b96324ea2-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "8e9e377b-388d-4e0a-b41f-a41b96324ea2" (UID: "8e9e377b-388d-4e0a-b41f-a41b96324ea2"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 16:53:19.806575 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:19.806557    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e9e377b-388d-4e0a-b41f-a41b96324ea2-kube-api-access-lb7rn" (OuterVolumeSpecName: "kube-api-access-lb7rn") pod "8e9e377b-388d-4e0a-b41f-a41b96324ea2" (UID: "8e9e377b-388d-4e0a-b41f-a41b96324ea2"). InnerVolumeSpecName "kube-api-access-lb7rn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 16:53:19.905166 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:19.905142    2579 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8e9e377b-388d-4e0a-b41f-a41b96324ea2-kube-rbac-proxy-sar-config\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:53:19.905166 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:19.905164    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lb7rn\" (UniqueName: \"kubernetes.io/projected/8e9e377b-388d-4e0a-b41f-a41b96324ea2-kube-api-access-lb7rn\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:53:19.905316 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:19.905173    2579 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8e9e377b-388d-4e0a-b41f-a41b96324ea2-cabundle-cert\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:53:19.905316 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:19.905182    2579 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8e9e377b-388d-4e0a-b41f-a41b96324ea2-proxy-tls\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:53:19.905316 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:19.905189    2579 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8e9e377b-388d-4e0a-b41f-a41b96324ea2-kserve-provision-location\") on node \"ip-10-0-141-35.ec2.internal\" DevicePath \"\""
Sep 15 16:53:20.589973 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:20.589945    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv" event={"ID":"8e9e377b-388d-4e0a-b41f-a41b96324ea2","Type":"ContainerDied","Data":"4f059a9ccd86d9b261de1b0ef5ea6dd5e9e4a4320219a161b2461cc565c4d145"}
Sep 15 16:53:20.590361 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:20.590033    2579 scope.go:117] "RemoveContainer" containerID="24ecc6a7d88670e3d9772dc021f042c0525ea19c4d58740325ebb48f135bdd64"
Sep 15 16:53:20.590361 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:20.589956    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv"
Sep 15 16:53:20.597517 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:20.597502    2579 scope.go:117] "RemoveContainer" containerID="e2b01b4e1cbe17c7727401eba1eaf0733225b1cd29095458b16deba6addf8b72"
Sep 15 16:53:20.604388 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:20.604368    2579 scope.go:117] "RemoveContainer" containerID="eb1d507ccee2a50282e8794b65976c56ee93f5448c2b6b9943f17aab64bd72cc"
Sep 15 16:53:20.607767 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:20.607746    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv"]
Sep 15 16:53:20.611373 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:20.611355    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-d567b8dc5-g8bqv"]
Sep 15 16:53:21.908844 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:53:21.908813    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e9e377b-388d-4e0a-b41f-a41b96324ea2" path="/var/lib/kubelet/pods/8e9e377b-388d-4e0a-b41f-a41b96324ea2/volumes"
Sep 15 16:55:58.371195 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:55:58.371162    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9hbcw_d1e544d2-21e5-4abd-82d9-33a82470f13c/global-pull-secret-syncer/0.log"
Sep 15 16:55:58.522328 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:55:58.522307    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-s7d5t_4893f9dd-7ff1-419a-9e27-c12efdb8285d/konnectivity-agent/0.log"
Sep 15 16:55:58.595559 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:55:58.595534    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-35.ec2.internal_269bca4735829fdfafb55c4d16c2e698/haproxy/0.log"
Sep 15 16:56:02.379955 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:02.379931    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bb3350d5-70d9-4dcd-a871-b74b9058f15a/alertmanager/0.log"
Sep 15 16:56:02.422049 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:02.422027    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bb3350d5-70d9-4dcd-a871-b74b9058f15a/config-reloader/0.log"
Sep 15 16:56:02.446917 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:02.446896    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bb3350d5-70d9-4dcd-a871-b74b9058f15a/kube-rbac-proxy-web/0.log"
Sep 15 16:56:02.478916 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:02.478892    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bb3350d5-70d9-4dcd-a871-b74b9058f15a/kube-rbac-proxy/0.log"
Sep 15 16:56:02.515298 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:02.515279    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bb3350d5-70d9-4dcd-a871-b74b9058f15a/kube-rbac-proxy-metric/0.log"
Sep 15 16:56:02.549000 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:02.548981    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bb3350d5-70d9-4dcd-a871-b74b9058f15a/prom-label-proxy/0.log"
Sep 15 16:56:02.575514 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:02.575494    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bb3350d5-70d9-4dcd-a871-b74b9058f15a/init-config-reloader/0.log"
Sep 15 16:56:02.950058 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:02.950039    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d2kqd_c7b5b20f-7476-4856-8cad-870d59f56499/node-exporter/0.log"
Sep 15 16:56:02.973772 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:02.973754    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d2kqd_c7b5b20f-7476-4856-8cad-870d59f56499/kube-rbac-proxy/0.log"
Sep 15 16:56:02.995581 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:02.995565    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d2kqd_c7b5b20f-7476-4856-8cad-870d59f56499/init-textfile/0.log"
Sep 15 16:56:03.128579 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:03.128558    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-g52bv_37670208-8bed-4d26-b9c2-069b51c3db6d/kube-rbac-proxy-main/0.log"
Sep 15 16:56:03.155561 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:03.155543    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-g52bv_37670208-8bed-4d26-b9c2-069b51c3db6d/kube-rbac-proxy-self/0.log"
Sep 15 16:56:03.176878 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:03.176856    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-g52bv_37670208-8bed-4d26-b9c2-069b51c3db6d/openshift-state-metrics/0.log"
Sep 15 16:56:03.495066 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:03.495040    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b848cd67c-kqz4k_a8e15716-7df1-488d-8adc-44a27496147f/telemeter-client/0.log"
Sep 15 16:56:03.518555 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:03.518536    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b848cd67c-kqz4k_a8e15716-7df1-488d-8adc-44a27496147f/reload/0.log"
Sep 15 16:56:03.545129 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:03.545110    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b848cd67c-kqz4k_a8e15716-7df1-488d-8adc-44a27496147f/kube-rbac-proxy/0.log"
Sep 15 16:56:03.577668 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:03.577651    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-759d5984cf-cx47r_763cda34-2afd-45fd-8b1d-c23899dfb0b2/thanos-query/0.log"
Sep 15 16:56:03.600351 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:03.600335    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-759d5984cf-cx47r_763cda34-2afd-45fd-8b1d-c23899dfb0b2/kube-rbac-proxy-web/0.log"
Sep 15 16:56:03.624575 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:03.624554    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-759d5984cf-cx47r_763cda34-2afd-45fd-8b1d-c23899dfb0b2/kube-rbac-proxy/0.log"
Sep 15 16:56:03.646492 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:03.646467    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-759d5984cf-cx47r_763cda34-2afd-45fd-8b1d-c23899dfb0b2/prom-label-proxy/0.log"
Sep 15 16:56:03.670119 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:03.670093    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-759d5984cf-cx47r_763cda34-2afd-45fd-8b1d-c23899dfb0b2/kube-rbac-proxy-rules/0.log"
Sep 15 16:56:03.691792 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:03.691765    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-759d5984cf-cx47r_763cda34-2afd-45fd-8b1d-c23899dfb0b2/kube-rbac-proxy-metrics/0.log"
Sep 15 16:56:05.974321 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:05.974296    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-75667bb54d-bw6pp_07f7ed38-cd70-409b-a08e-5449c1829b42/console/0.log"
Sep 15 16:56:06.553952 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.553921    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bwj65/perf-node-gather-daemonset-9d4pf"]
Sep 15 16:56:06.554230 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.554218    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c27f8f0-b46d-419c-8af9-3e81ac545e9d" containerName="storage-initializer"
Sep 15 16:56:06.554310 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.554231    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c27f8f0-b46d-419c-8af9-3e81ac545e9d" containerName="storage-initializer"
Sep 15 16:56:06.554310 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.554246    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e9e377b-388d-4e0a-b41f-a41b96324ea2" containerName="kserve-container"
Sep 15 16:56:06.554310 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.554251    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e9e377b-388d-4e0a-b41f-a41b96324ea2" containerName="kserve-container"
Sep 15 16:56:06.554310 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.554258    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c27f8f0-b46d-419c-8af9-3e81ac545e9d" containerName="kserve-container"
Sep 15 16:56:06.554310 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.554263    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c27f8f0-b46d-419c-8af9-3e81ac545e9d" containerName="kserve-container"
Sep 15 16:56:06.554310 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.554270    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e9e377b-388d-4e0a-b41f-a41b96324ea2" containerName="storage-initializer"
Sep 15 16:56:06.554310 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.554275    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e9e377b-388d-4e0a-b41f-a41b96324ea2" containerName="storage-initializer"
Sep 15 16:56:06.554310 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.554292    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c27f8f0-b46d-419c-8af9-3e81ac545e9d" containerName="kube-rbac-proxy"
Sep 15 16:56:06.554310 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.554296    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c27f8f0-b46d-419c-8af9-3e81ac545e9d" containerName="kube-rbac-proxy"
Sep 15 16:56:06.554310 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.554302    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e9e377b-388d-4e0a-b41f-a41b96324ea2" containerName="kube-rbac-proxy"
Sep 15 16:56:06.554310 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.554307    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e9e377b-388d-4e0a-b41f-a41b96324ea2" containerName="kube-rbac-proxy"
Sep 15 16:56:06.554669 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.554351    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e9e377b-388d-4e0a-b41f-a41b96324ea2" containerName="kserve-container"
Sep 15 16:56:06.554669 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.554358    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e9e377b-388d-4e0a-b41f-a41b96324ea2" containerName="kube-rbac-proxy"
Sep 15 16:56:06.554669 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.554366    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c27f8f0-b46d-419c-8af9-3e81ac545e9d" containerName="kserve-container"
Sep 15 16:56:06.554669 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.554372    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c27f8f0-b46d-419c-8af9-3e81ac545e9d" containerName="kube-rbac-proxy"
Sep 15 16:56:06.557290 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.557276    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bwj65/perf-node-gather-daemonset-9d4pf"
Sep 15 16:56:06.559591 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.559571    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bwj65\"/\"openshift-service-ca.crt\""
Sep 15 16:56:06.560651 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.560635    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bwj65\"/\"kube-root-ca.crt\""
Sep 15 16:56:06.560735 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.560635    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bwj65\"/\"default-dockercfg-l5mqt\""
Sep 15 16:56:06.565027 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.565004    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bwj65/perf-node-gather-daemonset-9d4pf"]
Sep 15 16:56:06.691099 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.691076    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6rkd\" (UniqueName: \"kubernetes.io/projected/4b9a5014-61a9-4797-82b0-e23d5d641b30-kube-api-access-l6rkd\") pod \"perf-node-gather-daemonset-9d4pf\" (UID: \"4b9a5014-61a9-4797-82b0-e23d5d641b30\") " pod="openshift-must-gather-bwj65/perf-node-gather-daemonset-9d4pf"
Sep 15 16:56:06.691206 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.691112    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4b9a5014-61a9-4797-82b0-e23d5d641b30-proc\") pod \"perf-node-gather-daemonset-9d4pf\" (UID: \"4b9a5014-61a9-4797-82b0-e23d5d641b30\") " pod="openshift-must-gather-bwj65/perf-node-gather-daemonset-9d4pf"
Sep 15 16:56:06.691206 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.691136    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b9a5014-61a9-4797-82b0-e23d5d641b30-sys\") pod \"perf-node-gather-daemonset-9d4pf\" (UID: \"4b9a5014-61a9-4797-82b0-e23d5d641b30\") " pod="openshift-must-gather-bwj65/perf-node-gather-daemonset-9d4pf"
Sep 15 16:56:06.691206 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.691154    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4b9a5014-61a9-4797-82b0-e23d5d641b30-lib-modules\") pod \"perf-node-gather-daemonset-9d4pf\" (UID: \"4b9a5014-61a9-4797-82b0-e23d5d641b30\") " pod="openshift-must-gather-bwj65/perf-node-gather-daemonset-9d4pf"
Sep 15 16:56:06.691206 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.691175    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4b9a5014-61a9-4797-82b0-e23d5d641b30-podres\") pod \"perf-node-gather-daemonset-9d4pf\" (UID: \"4b9a5014-61a9-4797-82b0-e23d5d641b30\") " pod="openshift-must-gather-bwj65/perf-node-gather-daemonset-9d4pf"
Sep 15 16:56:06.792248 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.792224    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6rkd\" (UniqueName: \"kubernetes.io/projected/4b9a5014-61a9-4797-82b0-e23d5d641b30-kube-api-access-l6rkd\") pod \"perf-node-gather-daemonset-9d4pf\" (UID: \"4b9a5014-61a9-4797-82b0-e23d5d641b30\") " pod="openshift-must-gather-bwj65/perf-node-gather-daemonset-9d4pf"
Sep 15 16:56:06.792345 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.792260    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4b9a5014-61a9-4797-82b0-e23d5d641b30-proc\") pod \"perf-node-gather-daemonset-9d4pf\" (UID: \"4b9a5014-61a9-4797-82b0-e23d5d641b30\") " pod="openshift-must-gather-bwj65/perf-node-gather-daemonset-9d4pf"
Sep 15 16:56:06.792345 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.792281    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b9a5014-61a9-4797-82b0-e23d5d641b30-sys\") pod \"perf-node-gather-daemonset-9d4pf\" (UID: \"4b9a5014-61a9-4797-82b0-e23d5d641b30\") " pod="openshift-must-gather-bwj65/perf-node-gather-daemonset-9d4pf"
Sep 15 16:56:06.792345 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.792297    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4b9a5014-61a9-4797-82b0-e23d5d641b30-lib-modules\") pod \"perf-node-gather-daemonset-9d4pf\" (UID: \"4b9a5014-61a9-4797-82b0-e23d5d641b30\") " pod="openshift-must-gather-bwj65/perf-node-gather-daemonset-9d4pf"
Sep 15 16:56:06.792504 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.792362    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4b9a5014-61a9-4797-82b0-e23d5d641b30-proc\") pod \"perf-node-gather-daemonset-9d4pf\" (UID: \"4b9a5014-61a9-4797-82b0-e23d5d641b30\") " pod="openshift-must-gather-bwj65/perf-node-gather-daemonset-9d4pf"
Sep 15 16:56:06.792504 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.792362    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b9a5014-61a9-4797-82b0-e23d5d641b30-sys\") pod \"perf-node-gather-daemonset-9d4pf\" (UID: \"4b9a5014-61a9-4797-82b0-e23d5d641b30\") " pod="openshift-must-gather-bwj65/perf-node-gather-daemonset-9d4pf"
Sep 15 16:56:06.792504 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.792384    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4b9a5014-61a9-4797-82b0-e23d5d641b30-podres\") pod \"perf-node-gather-daemonset-9d4pf\" (UID: \"4b9a5014-61a9-4797-82b0-e23d5d641b30\") " pod="openshift-must-gather-bwj65/perf-node-gather-daemonset-9d4pf"
Sep 15 16:56:06.792504 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.792435    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4b9a5014-61a9-4797-82b0-e23d5d641b30-podres\") pod \"perf-node-gather-daemonset-9d4pf\" (UID: \"4b9a5014-61a9-4797-82b0-e23d5d641b30\") " pod="openshift-must-gather-bwj65/perf-node-gather-daemonset-9d4pf"
Sep 15 16:56:06.792504 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.792391    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4b9a5014-61a9-4797-82b0-e23d5d641b30-lib-modules\") pod \"perf-node-gather-daemonset-9d4pf\" (UID: \"4b9a5014-61a9-4797-82b0-e23d5d641b30\") " pod="openshift-must-gather-bwj65/perf-node-gather-daemonset-9d4pf"
Sep 15 16:56:06.800183 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.800158    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6rkd\" (UniqueName: \"kubernetes.io/projected/4b9a5014-61a9-4797-82b0-e23d5d641b30-kube-api-access-l6rkd\") pod \"perf-node-gather-daemonset-9d4pf\" (UID: \"4b9a5014-61a9-4797-82b0-e23d5d641b30\") " pod="openshift-must-gather-bwj65/perf-node-gather-daemonset-9d4pf"
Sep 15 16:56:06.867110 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:06.867056    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bwj65/perf-node-gather-daemonset-9d4pf"
Sep 15 16:56:07.137880 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:07.137851    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bwj65/perf-node-gather-daemonset-9d4pf"]
Sep 15 16:56:07.140222 ip-10-0-141-35 kubenswrapper[2579]: W0915 16:56:07.140200    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4b9a5014_61a9_4797_82b0_e23d5d641b30.slice/crio-9447fdb6c1bc7e855118c67cd01f6e0a926f22a9ecca06b65aa92b626e482acd WatchSource:0}: Error finding container 9447fdb6c1bc7e855118c67cd01f6e0a926f22a9ecca06b65aa92b626e482acd: Status 404 returned error can't find the container with id 9447fdb6c1bc7e855118c67cd01f6e0a926f22a9ecca06b65aa92b626e482acd
Sep 15 16:56:07.268333 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:07.268304    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bwj65/perf-node-gather-daemonset-9d4pf" event={"ID":"4b9a5014-61a9-4797-82b0-e23d5d641b30","Type":"ContainerStarted","Data":"0cdb65b5e033482a76bc60afe9bcdab2ef7b70af6f9c5a17939083f79bde6332"}
Sep 15 16:56:07.268436 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:07.268353    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bwj65/perf-node-gather-daemonset-9d4pf" event={"ID":"4b9a5014-61a9-4797-82b0-e23d5d641b30","Type":"ContainerStarted","Data":"9447fdb6c1bc7e855118c67cd01f6e0a926f22a9ecca06b65aa92b626e482acd"}
Sep 15 16:56:07.268436 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:07.268389    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bwj65/perf-node-gather-daemonset-9d4pf"
Sep 15 16:56:07.284073 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:07.284032    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bwj65/perf-node-gather-daemonset-9d4pf" podStartSLOduration=1.284017221 podStartE2EDuration="1.284017221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 16:56:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 16:56:07.282681654 +0000 UTC m=+1783.912336005" watchObservedRunningTime="2026-09-15 16:56:07.284017221 +0000 UTC m=+1783.913671573"
Sep 15 16:56:07.397534 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:07.397482    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-45s5g_8d780ae3-adfc-447b-9605-30793d9fc5c1/dns/0.log"
Sep 15 16:56:07.417749 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:07.417731    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-45s5g_8d780ae3-adfc-447b-9605-30793d9fc5c1/kube-rbac-proxy/0.log"
Sep 15 16:56:07.540848 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:07.540826    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-btdbz_fdb8314d-7223-4926-b551-8bb1701052fd/dns-node-resolver/0.log"
Sep 15 16:56:08.037258 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:08.037236    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-8565fb9966-gk2rg_0aedf5a4-039b-42c4-8659-bc6e047eebe0/registry/0.log"
Sep 15 16:56:08.064170 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:08.064147    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8zk2f_f89021a6-bc8a-4f31-a76c-87c39f332c20/node-ca/0.log"
Sep 15 16:56:09.139960 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:09.139943    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2kxsk_84da0f72-32d0-4f59-9cad-f2c59cc71a0b/serve-healthcheck-canary/0.log"
Sep 15 16:56:09.552239 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:09.552215    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-27hdl_abc7e760-ef1c-48bd-8d65-c5ad19f3f422/kube-rbac-proxy/0.log"
Sep 15 16:56:09.573739 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:09.573723    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-27hdl_abc7e760-ef1c-48bd-8d65-c5ad19f3f422/exporter/0.log"
Sep 15 16:56:09.596479 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:09.596463    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-27hdl_abc7e760-ef1c-48bd-8d65-c5ad19f3f422/extractor/0.log"
Sep 15 16:56:11.896637 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:11.896608    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-b589c8cc8-gntnd_c61a1eff-9292-4700-a2c7-deb93bf22e86/manager/0.log"
Sep 15 16:56:11.923150 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:11.923128    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-645f9c8fd6-gfxnj_05583e14-ad32-4f6a-a34c-4b1d819da845/manager/0.log"
Sep 15 16:56:13.281922 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:13.281893    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bwj65/perf-node-gather-daemonset-9d4pf"
Sep 15 16:56:17.463121 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:17.463098    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cvgk4_b17d9664-4ce9-449a-8826-af717fd47f97/kube-multus-additional-cni-plugins/0.log"
Sep 15 16:56:17.486828 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:17.486805    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cvgk4_b17d9664-4ce9-449a-8826-af717fd47f97/egress-router-binary-copy/0.log"
Sep 15 16:56:17.509686 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:17.509667    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cvgk4_b17d9664-4ce9-449a-8826-af717fd47f97/cni-plugins/0.log"
Sep 15 16:56:17.529334 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:17.529310    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cvgk4_b17d9664-4ce9-449a-8826-af717fd47f97/bond-cni-plugin/0.log"
Sep 15 16:56:17.548571 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:17.548550    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cvgk4_b17d9664-4ce9-449a-8826-af717fd47f97/routeoverride-cni/0.log"
Sep 15 16:56:17.567159 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:17.567139    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cvgk4_b17d9664-4ce9-449a-8826-af717fd47f97/whereabouts-cni-bincopy/0.log"
Sep 15 16:56:17.586647 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:17.586629    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cvgk4_b17d9664-4ce9-449a-8826-af717fd47f97/whereabouts-cni/0.log"
Sep 15 16:56:18.015847 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:18.015771    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-dqm7s_ba7071d8-d459-4c22-ae2f-1ced3f8871ce/kube-multus/0.log"
Sep 15 16:56:18.201340 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:18.201319    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-txzqk_1d768ac0-1487-4850-bd9b-c66f73533e97/network-metrics-daemon/0.log"
Sep 15 16:56:18.230068 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:18.230045    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-txzqk_1d768ac0-1487-4850-bd9b-c66f73533e97/kube-rbac-proxy/0.log"
Sep 15 16:56:19.180986 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:19.180963    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4rdn_44ca88f0-c149-443e-a2d1-a30964775818/ovn-controller/0.log"
Sep 15 16:56:19.244985 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:19.244962    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4rdn_44ca88f0-c149-443e-a2d1-a30964775818/ovn-acl-logging/1.log"
Sep 15 16:56:19.245114 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:19.244990    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4rdn_44ca88f0-c149-443e-a2d1-a30964775818/ovn-acl-logging/0.log"
Sep 15 16:56:19.279810 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:19.279776    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4rdn_44ca88f0-c149-443e-a2d1-a30964775818/kube-rbac-proxy-node/0.log"
Sep 15 16:56:19.301275 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:19.301253    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4rdn_44ca88f0-c149-443e-a2d1-a30964775818/kube-rbac-proxy-ovn-metrics/0.log"
Sep 15 16:56:19.321148 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:19.321113    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4rdn_44ca88f0-c149-443e-a2d1-a30964775818/northd/0.log"
Sep 15 16:56:19.341309 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:19.341292    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4rdn_44ca88f0-c149-443e-a2d1-a30964775818/nbdb/0.log"
Sep 15 16:56:19.364927 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:19.364909    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4rdn_44ca88f0-c149-443e-a2d1-a30964775818/sbdb/0.log"
Sep 15 16:56:19.465709 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:19.465689    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4rdn_44ca88f0-c149-443e-a2d1-a30964775818/ovnkube-controller/0.log"
Sep 15 16:56:20.700866 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:20.700840    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-sqdn2_ad430be2-7c5f-4402-a5a7-c8b4359b8c02/network-check-target-container/0.log"
Sep 15 16:56:21.600647 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:21.600620    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-cs7tq_bf4e438a-fbc7-401f-b97c-bbb7bc4a0495/iptables-alerter/0.log"
Sep 15 16:56:22.215204 ip-10-0-141-35 kubenswrapper[2579]: I0915 16:56:22.215177    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-zr9w5_7a83b81c-a949-41e4-9e47-e7305b24d4a7/tuned/0.log"