Jul 01 14:37:06.164311 ip-10-0-139-55 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 01 14:37:06.164324 ip-10-0-139-55 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 01 14:37:06.164331 ip-10-0-139-55 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 01 14:37:06.164590 ip-10-0-139-55 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 01 14:37:16.208224 ip-10-0-139-55 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 01 14:37:16.208243 ip-10-0-139-55 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot e49a27afaab245ee81069e02c9eed361 --
Jul 01 14:39:21.240811 ip-10-0-139-55 systemd[1]: Starting Kubernetes Kubelet...
Jul 01 14:39:21.621732 ip-10-0-139-55 kubenswrapper[2551]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 01 14:39:21.621732 ip-10-0-139-55 kubenswrapper[2551]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 01 14:39:21.621732 ip-10-0-139-55 kubenswrapper[2551]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 01 14:39:21.621732 ip-10-0-139-55 kubenswrapper[2551]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 01 14:39:21.621732 ip-10-0-139-55 kubenswrapper[2551]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 01 14:39:21.623146 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.623056    2551 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 01 14:39:21.627803 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627785    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 14:39:21.627803 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627802    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 14:39:21.627870 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627806    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 14:39:21.627870 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627809    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 14:39:21.627870 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627813    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 14:39:21.627870 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627816    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 14:39:21.627870 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627818    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 14:39:21.627870 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627821    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 14:39:21.627870 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627825    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 14:39:21.627870 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627828    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 14:39:21.627870 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627831    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 14:39:21.627870 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627834    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 14:39:21.627870 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627842    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 14:39:21.627870 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627845    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 14:39:21.627870 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627848    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 14:39:21.627870 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627851    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 14:39:21.627870 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627853    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 14:39:21.627870 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627856    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 14:39:21.627870 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627859    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 14:39:21.627870 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627861    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 14:39:21.627870 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627864    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 14:39:21.627870 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627866    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 14:39:21.628334 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627869    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 14:39:21.628334 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627872    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 14:39:21.628334 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627875    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 14:39:21.628334 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627878    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 14:39:21.628334 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627881    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 14:39:21.628334 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627885    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 14:39:21.628334 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627887    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 14:39:21.628334 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627890    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 14:39:21.628334 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627892    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 14:39:21.628334 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627895    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 14:39:21.628334 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627897    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 14:39:21.628334 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627900    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 14:39:21.628334 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627903    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 14:39:21.628334 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627905    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 14:39:21.628334 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627908    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 14:39:21.628334 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627910    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 14:39:21.628334 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627913    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 14:39:21.628334 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627916    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 14:39:21.628334 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627918    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 14:39:21.628843 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627921    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 14:39:21.628843 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627923    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 14:39:21.628843 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627925    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 14:39:21.628843 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627928    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 14:39:21.628843 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627930    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 14:39:21.628843 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627933    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 14:39:21.628843 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627935    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 14:39:21.628843 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627937    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 14:39:21.628843 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627940    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 14:39:21.628843 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627943    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 14:39:21.628843 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627945    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 14:39:21.628843 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627948    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 14:39:21.628843 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627951    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 14:39:21.628843 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627954    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 14:39:21.628843 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627956    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 14:39:21.628843 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627959    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 14:39:21.628843 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627962    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 14:39:21.628843 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627964    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 14:39:21.628843 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627967    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 14:39:21.628843 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627969    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 14:39:21.629308 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627972    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 14:39:21.629308 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627974    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 14:39:21.629308 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627977    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 14:39:21.629308 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627980    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 14:39:21.629308 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627982    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 14:39:21.629308 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627984    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 14:39:21.629308 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627987    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 14:39:21.629308 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627989    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 14:39:21.629308 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627992    2551 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 14:39:21.629308 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.627996    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 14:39:21.629308 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628002    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 14:39:21.629308 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628006    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 14:39:21.629308 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628009    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 14:39:21.629308 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628012    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 14:39:21.629308 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628015    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 14:39:21.629308 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628018    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 14:39:21.629308 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628021    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 14:39:21.629308 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628024    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 14:39:21.629308 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628027    2551 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 14:39:21.629308 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628030    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 14:39:21.629787 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628033    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 14:39:21.629787 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628035    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 14:39:21.629787 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628038    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 14:39:21.629787 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628040    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 14:39:21.629787 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628043    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 14:39:21.629787 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628047    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 14:39:21.629787 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628050    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 14:39:21.629787 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628053    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 14:39:21.629787 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628055    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 14:39:21.629787 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628058    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 14:39:21.629787 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628062    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 14:39:21.629787 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628065    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 14:39:21.629787 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628068    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 14:39:21.629787 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628071    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 14:39:21.629787 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628074    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 14:39:21.629787 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628077    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 14:39:21.629787 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628079    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 14:39:21.629787 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628082    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 14:39:21.629787 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628843    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 14:39:21.630233 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628850    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 14:39:21.630233 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628853    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 14:39:21.630233 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628856    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 14:39:21.630233 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628859    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 14:39:21.630233 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628862    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 14:39:21.630233 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628867    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 14:39:21.630233 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628871    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 14:39:21.630233 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628875    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 14:39:21.630233 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628878    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 14:39:21.630233 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628880    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 14:39:21.630233 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628883    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 14:39:21.630233 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628886    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 14:39:21.630233 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628889    2551 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 14:39:21.630233 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628891    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 14:39:21.630233 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628894    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 14:39:21.630233 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628897    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 14:39:21.630233 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628900    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 14:39:21.630233 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628903    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 14:39:21.630233 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628906    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 14:39:21.630233 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628909    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 14:39:21.630728 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628913    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 14:39:21.630728 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628917    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 14:39:21.630728 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628921    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 14:39:21.630728 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628925    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 14:39:21.630728 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628931    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 14:39:21.630728 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628934    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 14:39:21.630728 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628938    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 14:39:21.630728 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628942    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 14:39:21.630728 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628946    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 14:39:21.630728 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628948    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 14:39:21.630728 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628951    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 14:39:21.630728 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628954    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 14:39:21.630728 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628957    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 14:39:21.630728 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628960    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 14:39:21.630728 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628964    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 14:39:21.630728 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628967    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 14:39:21.630728 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628970    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 14:39:21.630728 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628974    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 14:39:21.630728 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628976    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 14:39:21.631181 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628979    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 14:39:21.631181 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628981    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 14:39:21.631181 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628984    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 14:39:21.631181 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628987    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 14:39:21.631181 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628990    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 14:39:21.631181 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628993    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 14:39:21.631181 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628995    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 14:39:21.631181 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.628997    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 14:39:21.631181 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629000    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 14:39:21.631181 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629002    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 14:39:21.631181 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629005    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 14:39:21.631181 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629008    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 14:39:21.631181 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629011    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 14:39:21.631181 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629013    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 14:39:21.631181 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629016    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 14:39:21.631181 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629019    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 14:39:21.631181 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629022    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 14:39:21.631181 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629024    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 14:39:21.631181 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629027    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 14:39:21.631181 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629029    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 14:39:21.631722 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629032    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 14:39:21.631722 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629034    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 14:39:21.631722 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629037    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 14:39:21.631722 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629039    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 14:39:21.631722 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629042    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 14:39:21.631722 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629044    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 14:39:21.631722 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629047    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 14:39:21.631722 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629049    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 14:39:21.631722 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629052    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 14:39:21.631722 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629055    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 14:39:21.631722 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629057    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 14:39:21.631722 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629060    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 14:39:21.631722 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629063    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 14:39:21.631722 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629065    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 14:39:21.631722 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629068    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 14:39:21.631722 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629071    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 14:39:21.631722 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629074    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 14:39:21.631722 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629076    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 14:39:21.631722 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629079    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 14:39:21.631722 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629083    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 14:39:21.632195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629086    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 14:39:21.632195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629088    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 14:39:21.632195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629091    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 14:39:21.632195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629093    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 14:39:21.632195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629095    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 14:39:21.632195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629098    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 14:39:21.632195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629100    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 14:39:21.632195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629103    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 14:39:21.632195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629105    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 14:39:21.632195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629108    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 14:39:21.632195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629110    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 14:39:21.632195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629113    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 14:39:21.632195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629115    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 14:39:21.632195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629118    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 14:39:21.632195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629121    2551 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 14:39:21.632195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629124    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 14:39:21.632195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629126    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 14:39:21.632195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629130    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 14:39:21.632195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629133    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 14:39:21.632686 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629621    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 14:39:21.632686 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629627    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 14:39:21.632686 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629630    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 14:39:21.632686 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629635    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 14:39:21.632686 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629639    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 14:39:21.632686 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629643    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 14:39:21.632686 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629646    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 14:39:21.632686 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629650    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 14:39:21.632686 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629653    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 14:39:21.632686 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629655    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 14:39:21.632686 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629658    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 14:39:21.632686 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629661    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 14:39:21.632686 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629664    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 14:39:21.632686 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629667    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 14:39:21.632686 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629670    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 14:39:21.632686 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629672    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 14:39:21.632686 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629676    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 14:39:21.632686 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629679    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 14:39:21.632686 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629682    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 14:39:21.633153 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629685    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 14:39:21.633153 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629688    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 14:39:21.633153 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629691    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 14:39:21.633153 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629694    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 14:39:21.633153 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629697    2551 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 14:39:21.633153 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629700    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 14:39:21.633153 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629702    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 14:39:21.633153 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629705    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 14:39:21.633153 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629708    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 14:39:21.633153 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629711    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 14:39:21.633153 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629714    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 14:39:21.633153 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629717    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 14:39:21.633153 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629720    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 14:39:21.633153 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629722    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 14:39:21.633153 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629725    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 14:39:21.633153 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629727    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 14:39:21.633153 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629730    2551 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 14:39:21.633153 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629732    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 14:39:21.633153 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629735    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 14:39:21.633606 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629738    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 14:39:21.633606 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629742    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 14:39:21.633606 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629745    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 14:39:21.633606 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629747    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 14:39:21.633606 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629750    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 14:39:21.633606 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629753    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 14:39:21.633606 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629755    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 14:39:21.633606 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629758    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 14:39:21.633606 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629760    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 14:39:21.633606 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629762    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 14:39:21.633606 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629765    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 14:39:21.633606 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629767    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 14:39:21.633606 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629770    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 14:39:21.633606 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629772    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 14:39:21.633606 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629775    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 14:39:21.633606 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629777    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 14:39:21.633606 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629780    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 14:39:21.633606 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629782    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 14:39:21.633606 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629785    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 14:39:21.633606 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629788    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 14:39:21.634097 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629790    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 14:39:21.634097 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629793    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 14:39:21.634097 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629795    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 14:39:21.634097 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629799    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 14:39:21.634097 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629801    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 14:39:21.634097 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629804    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 14:39:21.634097 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629806    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 14:39:21.634097 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629809    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 14:39:21.634097 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629811    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 14:39:21.634097 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629814    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 14:39:21.634097 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629816    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 14:39:21.634097 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629819    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 14:39:21.634097 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629823    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 14:39:21.634097 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629825    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 14:39:21.634097 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629828    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 14:39:21.634097 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629830    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 14:39:21.634097 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629833    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 14:39:21.634097 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629835    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 14:39:21.634097 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629838    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 14:39:21.634097 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629841    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 14:39:21.634582 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629843    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 14:39:21.634582 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629846    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 14:39:21.634582 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629849    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 14:39:21.634582 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629851    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 14:39:21.634582 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629854    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 14:39:21.634582 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629857    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 14:39:21.634582 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629859    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 14:39:21.634582 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629862    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 14:39:21.634582 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629864    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 14:39:21.634582 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629867    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 14:39:21.634582 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629870    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 14:39:21.634582 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629872    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 14:39:21.634582 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629875    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 14:39:21.634582 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629877    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 14:39:21.634582 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629879    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 14:39:21.634582 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629882    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 14:39:21.634582 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629885    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 14:39:21.634582 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629887    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 14:39:21.634582 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629889    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 14:39:21.634582 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629892    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 14:39:21.635059 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.629894    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 14:39:21.635059 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.629967    2551 flags.go:64] FLAG: --address="0.0.0.0"
Jul 01 14:39:21.635059 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.629977    2551 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 01 14:39:21.635059 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.629997    2551 flags.go:64] FLAG: --anonymous-auth="true"
Jul 01 14:39:21.635059 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630004    2551 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 01 14:39:21.635059 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630011    2551 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 01 14:39:21.635059 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630015    2551 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 01 14:39:21.635059 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630020    2551 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 01 14:39:21.635059 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630028    2551 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 01 14:39:21.635059 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630031    2551 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 01 14:39:21.635059 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630035    2551 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 01 14:39:21.635059 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630038    2551 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 01 14:39:21.635059 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630041    2551 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 01 14:39:21.635059 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630045    2551 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 01 14:39:21.635059 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630048    2551 flags.go:64] FLAG: --cgroup-root=""
Jul 01 14:39:21.635059 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630052    2551 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 01 14:39:21.635059 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630055    2551 flags.go:64] FLAG: --client-ca-file=""
Jul 01 14:39:21.635059 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630058    2551 flags.go:64] FLAG: --cloud-provider="external"
Jul 01 14:39:21.635059 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630061    2551 flags.go:64] FLAG: --cluster-dns="[]"
Jul 01 14:39:21.635059 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630065    2551 flags.go:64] FLAG: --cluster-domain=""
Jul 01 14:39:21.635059 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630068    2551 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 01 14:39:21.635059 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630071    2551 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 01 14:39:21.635059 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630075    2551 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 01 14:39:21.635059 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630078    2551 flags.go:64] FLAG: --container-log-max-files="5"
Jul 01 14:39:21.635633 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630082    2551 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 01 14:39:21.635633 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630085    2551 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 01 14:39:21.635633 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630088    2551 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 01 14:39:21.635633 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630091    2551 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 01 14:39:21.635633 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630094    2551 flags.go:64] FLAG: --contention-profiling="false"
Jul 01 14:39:21.635633 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630097    2551 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 01 14:39:21.635633 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630100    2551 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 01 14:39:21.635633 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630103    2551 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 01 14:39:21.635633 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630106    2551 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 01 14:39:21.635633 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630111    2551 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 01 14:39:21.635633 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630115    2551 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 01 14:39:21.635633 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630118    2551 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 01 14:39:21.635633 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630121    2551 flags.go:64] FLAG: --enable-load-reader="false"
Jul 01 14:39:21.635633 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630124    2551 flags.go:64] FLAG: --enable-server="true"
Jul 01 14:39:21.635633 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630127    2551 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 01 14:39:21.635633 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630132    2551 flags.go:64] FLAG: --event-burst="100"
Jul 01 14:39:21.635633 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630135    2551 flags.go:64] FLAG: --event-qps="50"
Jul 01 14:39:21.635633 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630138    2551 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 01 14:39:21.635633 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630141    2551 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 01 14:39:21.635633 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630144    2551 flags.go:64] FLAG: --eviction-hard=""
Jul 01 14:39:21.635633 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630148    2551 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 01 14:39:21.635633 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630150    2551 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 01 14:39:21.635633 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630153    2551 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 01 14:39:21.635633 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630157    2551 flags.go:64] FLAG: --eviction-soft=""
Jul 01 14:39:21.635633 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630159    2551 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 01 14:39:21.636208 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630162    2551 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 01 14:39:21.636208 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630166    2551 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 01 14:39:21.636208 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630169    2551 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 01 14:39:21.636208 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630171    2551 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 01 14:39:21.636208 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630174    2551 flags.go:64] FLAG: --fail-swap-on="true"
Jul 01 14:39:21.636208 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630177    2551 flags.go:64] FLAG: --feature-gates=""
Jul 01 14:39:21.636208 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630181    2551 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 01 14:39:21.636208 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630185    2551 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 01 14:39:21.636208 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630188    2551 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 01 14:39:21.636208 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630191    2551 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 01 14:39:21.636208 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630194    2551 flags.go:64] FLAG: --healthz-port="10248"
Jul 01 14:39:21.636208 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630197    2551 flags.go:64] FLAG: --help="false"
Jul 01 14:39:21.636208 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630200    2551 flags.go:64] FLAG: --hostname-override="ip-10-0-139-55.ec2.internal"
Jul 01 14:39:21.636208 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630203    2551 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 01 14:39:21.636208 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630206    2551 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 01 14:39:21.636208 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630211    2551 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 01 14:39:21.636208 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630215    2551 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 01 14:39:21.636208 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630219    2551 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 01 14:39:21.636208 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630222    2551 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 01 14:39:21.636208 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630224    2551 flags.go:64] FLAG: --image-service-endpoint=""
Jul 01 14:39:21.636208 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630227    2551 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 01 14:39:21.636208 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630230    2551 flags.go:64] FLAG: --kube-api-burst="100"
Jul 01 14:39:21.636208 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630234    2551 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 01 14:39:21.636845 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630237    2551 flags.go:64] FLAG: --kube-api-qps="50"
Jul 01 14:39:21.636845 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630240    2551 flags.go:64] FLAG: --kube-reserved=""
Jul 01 14:39:21.636845 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630243    2551 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 01 14:39:21.636845 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630247    2551 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 01 14:39:21.636845 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630250    2551 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 01 14:39:21.636845 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630252    2551 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 01 14:39:21.636845 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630255    2551 flags.go:64] FLAG: --lock-file=""
Jul 01 14:39:21.636845 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630258    2551 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 01 14:39:21.636845 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630261    2551 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 01 14:39:21.636845 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630264    2551 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 01 14:39:21.636845 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630269    2551 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 01 14:39:21.636845 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630272    2551 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 01 14:39:21.636845 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630275    2551 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 01 14:39:21.636845 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630277    2551 flags.go:64] FLAG: --logging-format="text"
Jul 01 14:39:21.636845 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630280    2551 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 01 14:39:21.636845 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630284    2551 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 01 14:39:21.636845 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630286    2551 flags.go:64] FLAG: --manifest-url=""
Jul 01 14:39:21.636845 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630289    2551 flags.go:64] FLAG: --manifest-url-header=""
Jul 01 14:39:21.636845 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630293    2551 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 01 14:39:21.636845 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630296    2551 flags.go:64] FLAG: --max-open-files="1000000"
Jul 01 14:39:21.636845 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630301    2551 flags.go:64] FLAG: --max-pods="110"
Jul 01 14:39:21.636845 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630304    2551 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 01 14:39:21.636845 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630307    2551 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 01 14:39:21.636845 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630309    2551 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 01 14:39:21.636845 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630315    2551 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 01 14:39:21.637480 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630317    2551 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 01 14:39:21.637480 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630322    2551 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 01 14:39:21.637480 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630325    2551 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 01 14:39:21.637480 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630338    2551 flags.go:64] FLAG: --node-status-max-images="50"
Jul 01 14:39:21.637480 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630341    2551 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 01 14:39:21.637480 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630344    2551 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 01 14:39:21.637480 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630347    2551 flags.go:64] FLAG: --pod-cidr=""
Jul 01 14:39:21.637480 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630350    2551 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8a2632cd31d78133be7b5fb6999469b2acf9b98d077c7f6ae5d0d94083d76b04"
Jul 01 14:39:21.637480 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630357    2551 flags.go:64] FLAG: --pod-manifest-path=""
Jul 01 14:39:21.637480 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630360    2551 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 01 14:39:21.637480 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630363    2551 flags.go:64] FLAG: --pods-per-core="0"
Jul 01 14:39:21.637480 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630366    2551 flags.go:64] FLAG: --port="10250"
Jul 01 14:39:21.637480 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630369    2551 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 01 14:39:21.637480 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630372    2551 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b9c06b96afc82eb4"
Jul 01 14:39:21.637480 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630375    2551 flags.go:64] FLAG: --qos-reserved=""
Jul 01 14:39:21.637480 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630378    2551 flags.go:64] FLAG: --read-only-port="10255"
Jul 01 14:39:21.637480 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630381    2551 flags.go:64] FLAG: --register-node="true"
Jul 01 14:39:21.637480 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630384    2551 flags.go:64] FLAG: --register-with-taints=""
Jul 01 14:39:21.637480 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630388    2551 flags.go:64] FLAG: --registry-burst="10"
Jul 01 14:39:21.637480 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630390    2551 flags.go:64] FLAG: --registry-qps="5"
Jul 01 14:39:21.637480 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630393    2551 flags.go:64] FLAG: --reserved-cpus=""
Jul 01 14:39:21.637480 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630396    2551 flags.go:64] FLAG: --reserved-memory=""
Jul 01 14:39:21.637480 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630400    2551 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 01 14:39:21.637480 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630416    2551 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 01 14:39:21.637480 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630420    2551 flags.go:64] FLAG: --rotate-certificates="false"
Jul 01 14:39:21.638075 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630423    2551 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 01 14:39:21.638075 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630426    2551 flags.go:64] FLAG: --runonce="false"
Jul 01 14:39:21.638075 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630428    2551 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 01 14:39:21.638075 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630431    2551 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 01 14:39:21.638075 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630434    2551 flags.go:64] FLAG: --seccomp-default="false"
Jul 01 14:39:21.638075 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630437    2551 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 01 14:39:21.638075 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630442    2551 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 01 14:39:21.638075 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630445    2551 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 01 14:39:21.638075 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630448    2551 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 01 14:39:21.638075 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630453    2551 flags.go:64] FLAG: --storage-driver-password="root"
Jul 01 14:39:21.638075 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630456    2551 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 01 14:39:21.638075 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630459    2551 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 01 14:39:21.638075 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630462    2551 flags.go:64] FLAG: --storage-driver-user="root"
Jul 01 14:39:21.638075 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630465    2551 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 01 14:39:21.638075 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630468    2551 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 01 14:39:21.638075 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630472    2551 flags.go:64] FLAG: --system-cgroups=""
Jul 01 14:39:21.638075 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630475    2551 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 01 14:39:21.638075 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630480    2551 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 01 14:39:21.638075 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630483    2551 flags.go:64] FLAG: --tls-cert-file=""
Jul 01 14:39:21.638075 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630486    2551 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 01 14:39:21.638075 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630490    2551 flags.go:64] FLAG: --tls-min-version=""
Jul 01 14:39:21.638075 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630493    2551 flags.go:64] FLAG: --tls-private-key-file=""
Jul 01 14:39:21.638075 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630495    2551 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 01 14:39:21.638075 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630498    2551 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 01 14:39:21.638075 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630501    2551 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 01 14:39:21.638694 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630504    2551 flags.go:64] FLAG: --v="2"
Jul 01 14:39:21.638694 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630508    2551 flags.go:64] FLAG: --version="false"
Jul 01 14:39:21.638694 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630512    2551 flags.go:64] FLAG: --vmodule=""
Jul 01 14:39:21.638694 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630516    2551 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 01 14:39:21.638694 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.630519    2551 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 01 14:39:21.638694 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630644    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 14:39:21.638694 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630648    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 14:39:21.638694 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630651    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 14:39:21.638694 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630653    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 14:39:21.638694 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630656    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 14:39:21.638694 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630659    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 14:39:21.638694 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630661    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 14:39:21.638694 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630664    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 14:39:21.638694 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630669    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 14:39:21.638694 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630671    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 14:39:21.638694 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630674    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 14:39:21.638694 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630679    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 14:39:21.638694 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630682    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 14:39:21.638694 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630685    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 14:39:21.638694 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630687    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 14:39:21.638694 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630691    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 14:39:21.639195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630693    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 14:39:21.639195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630696    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 14:39:21.639195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630699    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 14:39:21.639195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630702    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 14:39:21.639195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630706    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 14:39:21.639195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630708    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 14:39:21.639195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630711    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 14:39:21.639195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630714    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 14:39:21.639195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630716    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 14:39:21.639195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630719    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 14:39:21.639195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630721    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 14:39:21.639195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630724    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 14:39:21.639195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630726    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 14:39:21.639195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630729    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 14:39:21.639195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630732    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 14:39:21.639195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630734    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 14:39:21.639195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630737    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 14:39:21.639195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630740    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 14:39:21.639195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630743    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 14:39:21.639195 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630746    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 14:39:21.639708 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630749    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 14:39:21.639708 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630751    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 14:39:21.639708 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630753    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 14:39:21.639708 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630757    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 14:39:21.639708 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630763    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 14:39:21.639708 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630766    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 14:39:21.639708 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630770    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 14:39:21.639708 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630775    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 14:39:21.639708 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630778    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 14:39:21.639708 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630781    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 14:39:21.639708 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630784    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 14:39:21.639708 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630787    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 14:39:21.639708 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630789    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 14:39:21.639708 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630792    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 14:39:21.639708 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630795    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 14:39:21.639708 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630798    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 14:39:21.639708 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630801    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 14:39:21.639708 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630804    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 14:39:21.640151 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630806    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 14:39:21.640151 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630809    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 14:39:21.640151 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630812    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 14:39:21.640151 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630814    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 14:39:21.640151 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630817    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 14:39:21.640151 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630821    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 14:39:21.640151 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630825    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 14:39:21.640151 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630828    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 14:39:21.640151 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630831    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 14:39:21.640151 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630833    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 14:39:21.640151 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630836    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 14:39:21.640151 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630839    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 14:39:21.640151 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630842    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 14:39:21.640151 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630845    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 14:39:21.640151 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630847    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 14:39:21.640151 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630850    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 14:39:21.640151 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630853    2551 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 14:39:21.640151 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630856    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 14:39:21.640151 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630860    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 14:39:21.640151 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630862    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 14:39:21.640696 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630865    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 14:39:21.640696 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630869    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 14:39:21.640696 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630871    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 14:39:21.640696 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630874    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 14:39:21.640696 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630876    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 14:39:21.640696 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630879    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 14:39:21.640696 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630882    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 14:39:21.640696 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630884    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 14:39:21.640696 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630887    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 14:39:21.640696 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630889    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 14:39:21.640696 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630892    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 14:39:21.640696 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630895    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 14:39:21.640696 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630897    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 14:39:21.640696 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630900    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 14:39:21.640696 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630902    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 14:39:21.640696 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630905    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 14:39:21.640696 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630907    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 14:39:21.640696 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630910    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 14:39:21.640696 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630912    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 14:39:21.640696 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630915    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 14:39:21.641179 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630917    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 14:39:21.641179 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630920    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 14:39:21.641179 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630923    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 14:39:21.641179 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630925    2551 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 14:39:21.641179 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.630928    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 14:39:21.641179 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.631757    2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 01 14:39:21.641179 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.639213    2551 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 01 14:39:21.641179 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.639233    2551 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 01 14:39:21.641179 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.639982    2551 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 01 14:39:21.641179 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.639991    2551 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 01 14:39:21.641179 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640059    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 14:39:21.641179 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640065    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 14:39:21.641179 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640068    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 14:39:21.641179 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640071    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 14:39:21.641179 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640074    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 14:39:21.641572 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640077    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 14:39:21.641572 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640080    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 14:39:21.641572 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640083    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 14:39:21.641572 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640088    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 14:39:21.641572 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640093    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 14:39:21.641572 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640096    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 14:39:21.641572 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640099    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 14:39:21.641572 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640102    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 14:39:21.641572 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640105    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 14:39:21.641572 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640108    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 14:39:21.641572 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640110    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 14:39:21.641572 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640113    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 14:39:21.641572 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640115    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 14:39:21.641572 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640118    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 14:39:21.641572 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640121    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 14:39:21.641572 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640123    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 14:39:21.641572 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640126    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 14:39:21.641572 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640129    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 14:39:21.641572 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640131    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 14:39:21.642013 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640134    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 14:39:21.642013 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640136    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 14:39:21.642013 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640139    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 14:39:21.642013 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640142    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 14:39:21.642013 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640144    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 14:39:21.642013 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640153    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 14:39:21.642013 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640157    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 14:39:21.642013 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640159    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 14:39:21.642013 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640162    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 14:39:21.642013 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640165    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 14:39:21.642013 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640168    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 14:39:21.642013 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640170    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 14:39:21.642013 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640173    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 14:39:21.642013 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640176    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 14:39:21.642013 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640178    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 14:39:21.642013 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640181    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 14:39:21.642013 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640183    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 14:39:21.642013 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640186    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 14:39:21.642013 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640188    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 14:39:21.642573 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640191    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 14:39:21.642573 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640195    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 14:39:21.642573 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640199    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 14:39:21.642573 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640201    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 14:39:21.642573 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640204    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 14:39:21.642573 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640207    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 14:39:21.642573 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640209    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 14:39:21.642573 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640211    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 14:39:21.642573 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640214    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 14:39:21.642573 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640216    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 14:39:21.642573 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640219    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 14:39:21.642573 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640222    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 14:39:21.642573 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640224    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 14:39:21.642573 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640227    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 14:39:21.642573 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640229    2551 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 14:39:21.642573 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640232    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 14:39:21.642573 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640234    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 14:39:21.642573 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640236    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 14:39:21.642573 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640241    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 14:39:21.643036 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640251    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 14:39:21.643036 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640253    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 14:39:21.643036 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640256    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 14:39:21.643036 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640259    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 14:39:21.643036 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640261    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 14:39:21.643036 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640264    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 14:39:21.643036 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640267    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 14:39:21.643036 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640269    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 14:39:21.643036 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640272    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 14:39:21.643036 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640274    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 14:39:21.643036 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640277    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 14:39:21.643036 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640279    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 14:39:21.643036 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640282    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 14:39:21.643036 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640286    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 14:39:21.643036 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640289    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 14:39:21.643036 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640292    2551 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 14:39:21.643036 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640295    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 14:39:21.643036 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640297    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 14:39:21.643036 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640300    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 14:39:21.643036 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640303    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 14:39:21.643535 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640306    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 14:39:21.643535 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640309    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 14:39:21.643535 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640311    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 14:39:21.643535 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640314    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 14:39:21.643535 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640317    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 14:39:21.643535 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640319    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 14:39:21.643535 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640322    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 14:39:21.643535 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640325    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 14:39:21.643535 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640327    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 14:39:21.643535 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640330    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 14:39:21.643535 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640333    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 14:39:21.643535 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640336    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 14:39:21.643535 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640338    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 14:39:21.643535 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640346    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 14:39:21.643535 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640349    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 14:39:21.643535 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640352    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 14:39:21.643535 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640354    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 14:39:21.643991 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.640359    2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 01 14:39:21.643991 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640525    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 14:39:21.643991 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640531    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 14:39:21.643991 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640534    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 14:39:21.643991 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640537    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 14:39:21.643991 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640540    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 14:39:21.643991 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640549    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 14:39:21.643991 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640554    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 14:39:21.643991 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640557    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 14:39:21.643991 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640559    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 14:39:21.643991 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640562    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 14:39:21.643991 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640564    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 14:39:21.643991 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640567    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 14:39:21.643991 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640570    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 14:39:21.643991 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640572    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 14:39:21.644348 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640575    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 14:39:21.644348 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640578    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 14:39:21.644348 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640580    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 14:39:21.644348 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640583    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 14:39:21.644348 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640586    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 14:39:21.644348 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640588    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 14:39:21.644348 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640591    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 14:39:21.644348 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640593    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 14:39:21.644348 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640596    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 14:39:21.644348 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640598    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 14:39:21.644348 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640601    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 14:39:21.644348 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640604    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 14:39:21.644348 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640608    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 14:39:21.644348 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640612    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 14:39:21.644348 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640621    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 14:39:21.644348 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640624    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 14:39:21.644348 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640626    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 14:39:21.644348 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640634    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 14:39:21.644348 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640637    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 14:39:21.644348 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640640    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 14:39:21.644847 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640642    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 14:39:21.644847 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640645    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 14:39:21.644847 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640647    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 14:39:21.644847 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640650    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 14:39:21.644847 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640652    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 14:39:21.644847 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640655    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 14:39:21.644847 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640657    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 14:39:21.644847 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640660    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 14:39:21.644847 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640662    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 14:39:21.644847 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640664    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 14:39:21.644847 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640667    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 14:39:21.644847 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640669    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 14:39:21.644847 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640672    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 14:39:21.644847 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640674    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 14:39:21.644847 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640676    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 14:39:21.644847 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640679    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 14:39:21.644847 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640681    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 14:39:21.644847 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640684    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 14:39:21.644847 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640686    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 14:39:21.644847 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640689    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 14:39:21.644847 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640691    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 14:39:21.645345 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640694    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 14:39:21.645345 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640697    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 14:39:21.645345 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640699    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 14:39:21.645345 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640702    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 14:39:21.645345 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640705    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 14:39:21.645345 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640708    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 14:39:21.645345 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640716    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 14:39:21.645345 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640719    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 14:39:21.645345 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640721    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 14:39:21.645345 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640724    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 14:39:21.645345 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640726    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 14:39:21.645345 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640729    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 14:39:21.645345 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640731    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 14:39:21.645345 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640733    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 14:39:21.645345 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640736    2551 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 14:39:21.645345 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640739    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 14:39:21.645345 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640741    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 14:39:21.645345 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640745    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 14:39:21.645345 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640748    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 14:39:21.645812 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640751    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 14:39:21.645812 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640754    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 14:39:21.645812 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640756    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 14:39:21.645812 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640759    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 14:39:21.645812 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640762    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 14:39:21.645812 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640765    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 14:39:21.645812 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640767    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 14:39:21.645812 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640770    2551 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 14:39:21.645812 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640772    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 14:39:21.645812 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640775    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 14:39:21.645812 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640777    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 14:39:21.645812 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640780    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 14:39:21.645812 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640783    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 14:39:21.645812 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640785    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 14:39:21.645812 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640788    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 14:39:21.645812 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640791    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 14:39:21.645812 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640793    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 14:39:21.645812 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640796    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 14:39:21.645812 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640798    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 14:39:21.646267 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640801    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 14:39:21.646267 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640809    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 14:39:21.646267 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640813    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 14:39:21.646267 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640816    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 14:39:21.646267 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640818    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 14:39:21.646267 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:21.640821    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 14:39:21.646267 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.640826    2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 01 14:39:21.646267 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.640965    2551 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 01 14:39:21.646267 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.643350    2551 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 01 14:39:21.646267 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.644886    2551 server.go:1019] "Starting client certificate rotation"
Jul 01 14:39:21.646267 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.645000    2551 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 01 14:39:21.646267 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.645060    2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 01 14:39:21.648516 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.648501    2551 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 01 14:39:21.655093 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.655072    2551 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 01 14:39:21.666768 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.666742    2551 log.go:25] "Validated CRI v1 runtime API"
Jul 01 14:39:21.669169 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.669152    2551 log.go:25] "Validated CRI v1 image API"
Jul 01 14:39:21.670025 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.670011    2551 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 01 14:39:21.674686 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.674660    2551 fs.go:135] Filesystem UUIDs: map[3a72ddf4-888e-48f4-94c7-0c147dd02cd5:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c9ab0e59-7734-46e0-86d2-fa62e776b610:/dev/nvme0n1p3]
Jul 01 14:39:21.674763 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.674685    2551 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 01 14:39:21.679681 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.679655    2551 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 01 14:39:21.680060 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.679937    2551 manager.go:216] Machine: {Timestamp:2026-07-01 14:39:21.678674724 +0000 UTC m=+0.319460679 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099343 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2469995dd89c022c34b7c94bdfde9c SystemUUID:ec246999-5dd8-9c02-2c34-b7c94bdfde9c BootID:e49a27af-aab2-45ee-8106-9e02c9eed361 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:59:17:aa:92:95 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:59:17:aa:92:95 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e2:11:88:49:ff:78 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 01 14:39:21.680060 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.680054    2551 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 01 14:39:21.680168 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.680145    2551 manager.go:232] Version: {KernelVersion:5.14.0-570.124.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260623-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 01 14:39:21.683622 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.683595    2551 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 01 14:39:21.683818 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.683625    2551 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-55.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 01 14:39:21.684493 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.684481    2551 topology_manager.go:138] "Creating topology manager with none policy"
Jul 01 14:39:21.684556 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.684500    2551 container_manager_linux.go:309] "Creating device plugin manager"
Jul 01 14:39:21.684556 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.684510    2551 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 01 14:39:21.685742 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.685729    2551 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 01 14:39:21.687494 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.687482    2551 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 01 14:39:21.690976 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.690960    2551 state_mem.go:36] "Initialized new in-memory state store"
Jul 01 14:39:21.691124 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.691112    2551 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 01 14:39:21.691181 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.691153    2551 kubelet.go:486] "Attempting to sync node with API server"
Jul 01 14:39:21.691181 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.691168    2551 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 01 14:39:21.691262 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.691183    2551 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 01 14:39:21.691262 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.691199    2551 kubelet.go:398] "Adding apiserver pod source"
Jul 01 14:39:21.691262 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.691211    2551 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 01 14:39:21.691667 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.691652    2551 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 01 14:39:21.691746 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.691676    2551 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 01 14:39:21.695192 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.695173    2551 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 01 14:39:21.696936 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.696921    2551 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 01 14:39:21.697007 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.696951    2551 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 01 14:39:21.697561 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.697550    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 01 14:39:21.697607 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.697567    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 01 14:39:21.697607 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.697573    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 01 14:39:21.697607 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.697579    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 01 14:39:21.697607 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.697591    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 01 14:39:21.697607 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.697598    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 01 14:39:21.697607 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.697607    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 01 14:39:21.697761 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.697612    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 01 14:39:21.697761 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.697620    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 01 14:39:21.697761 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.697625    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 01 14:39:21.697761 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.697634    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 01 14:39:21.698936 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.698913    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 01 14:39:21.699899 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.699505    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 01 14:39:21.699899 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.699523    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 01 14:39:21.703386 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.703362    2551 server.go:1268] "Started kubelet"
Jul 01 14:39:21.703570 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.703492    2551 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 01 14:39:21.703636 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.703594    2551 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 01 14:39:21.703636 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.703610    2551 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 01 14:39:21.704354 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.704332    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-55.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 01 14:39:21.704524 ip-10-0-139-55 systemd[1]: Started Kubernetes Kubelet.
Jul 01 14:39:21.704796 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:21.704768    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-55.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 01 14:39:21.704875 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:21.704774    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 01 14:39:21.705187 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.705163    2551 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 01 14:39:21.705907 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.705892    2551 server.go:310] "Adding debug handlers to kubelet server"
Jul 01 14:39:21.709106 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:21.709076    2551 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 01 14:39:21.710583 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:21.709545    2551 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-55.ec2.internal.18be3159356a7883  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-55.ec2.internal,UID:ip-10-0-139-55.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-55.ec2.internal,},FirstTimestamp:2026-07-01 14:39:21.703315587 +0000 UTC m=+0.344101549,LastTimestamp:2026-07-01 14:39:21.703315587 +0000 UTC m=+0.344101549,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-55.ec2.internal,}"
Jul 01 14:39:21.712640 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.712622    2551 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 01 14:39:21.712726 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.712670    2551 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 01 14:39:21.712726 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:21.712711    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-55.ec2.internal\" not found"
Jul 01 14:39:21.713368 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.713348    2551 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 01 14:39:21.713368 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.713350    2551 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 01 14:39:21.713512 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.713380    2551 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 01 14:39:21.713512 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.713484    2551 reconstruct.go:97] "Volume reconstruction finished"
Jul 01 14:39:21.713512 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.713492    2551 reconciler.go:29] "Reconciler: start to sync state"
Jul 01 14:39:21.715044 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.715027    2551 factory.go:55] Registering systemd factory
Jul 01 14:39:21.715044 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.715046    2551 factory.go:223] Registration of the systemd container factory successfully
Jul 01 14:39:21.715259 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.715245    2551 factory.go:153] Registering CRI-O factory
Jul 01 14:39:21.715259 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.715260    2551 factory.go:223] Registration of the crio container factory successfully
Jul 01 14:39:21.715378 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.715313    2551 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 01 14:39:21.715378 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.715338    2551 factory.go:103] Registering Raw factory
Jul 01 14:39:21.715378 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.715355    2551 manager.go:1196] Started watching for new ooms in manager
Jul 01 14:39:21.716025 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.716011    2551 manager.go:349] Starting recovery of all containers
Jul 01 14:39:21.717368 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:21.717316    2551 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-55.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 01 14:39:21.717464 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:21.717380    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 01 14:39:21.727754 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.727575    2551 manager.go:354] Recovery completed
Jul 01 14:39:21.732456 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.732429    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 14:39:21.732842 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.732824    2551 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cp8mq"
Jul 01 14:39:21.734700 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.734684    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 14:39:21.734748 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.734720    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 14:39:21.734748 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.734732    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasSufficientPID"
Jul 01 14:39:21.735298 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.735283    2551 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 01 14:39:21.735298 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.735296    2551 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 01 14:39:21.735370 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.735311    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 14:39:21.735399 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.735315    2551 state_mem.go:36] "Initialized new in-memory state store"
Jul 01 14:39:21.736918 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.736903    2551 policy_none.go:49] "None policy: Start"
Jul 01 14:39:21.736987 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.736921    2551 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 01 14:39:21.736987 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:21.736868    2551 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-55.ec2.internal.18be3159374962db  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-55.ec2.internal,UID:ip-10-0-139-55.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-55.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-55.ec2.internal,},FirstTimestamp:2026-07-01 14:39:21.734701787 +0000 UTC m=+0.375487749,LastTimestamp:2026-07-01 14:39:21.734701787 +0000 UTC m=+0.375487749,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-55.ec2.internal,}"
Jul 01 14:39:21.737109 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.736932    2551 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 01 14:39:21.738013 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.738002    2551 policy_none.go:47] "Start"
Jul 01 14:39:21.739912 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.739894    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 14:39:21.740023 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.739927    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 14:39:21.740023 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.739940    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasSufficientPID"
Jul 01 14:39:21.742114 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.742094    2551 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cp8mq"
Jul 01 14:39:21.771018 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.771000    2551 manager.go:337] "Starting Device Plugin manager"
Jul 01 14:39:21.789016 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:21.771040    2551 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 01 14:39:21.789016 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.771053    2551 server.go:87] "Starting device plugin registration server"
Jul 01 14:39:21.789016 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.771346    2551 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 01 14:39:21.789016 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.771365    2551 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 01 14:39:21.789016 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.771474    2551 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 01 14:39:21.789016 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.771589    2551 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 01 14:39:21.789016 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.771602    2551 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 01 14:39:21.789016 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:21.772131    2551 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 01 14:39:21.789016 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:21.772178    2551 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-55.ec2.internal\" not found"
Jul 01 14:39:21.789016 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.784053    2551 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 01 14:39:21.789016 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.785164    2551 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 01 14:39:21.789016 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.785211    2551 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 01 14:39:21.789016 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.785241    2551 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 01 14:39:21.789016 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:21.785287    2551 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 01 14:39:21.789016 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.787655    2551 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 14:39:21.871686 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.871649    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 14:39:21.872728 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.872678    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 14:39:21.872728 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.872711    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 14:39:21.872728 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.872721    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasSufficientPID"
Jul 01 14:39:21.872869 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.872747    2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-55.ec2.internal"
Jul 01 14:39:21.879452 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.879436    2551 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-55.ec2.internal"
Jul 01 14:39:21.879498 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:21.879462    2551 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-55.ec2.internal\": node \"ip-10-0-139-55.ec2.internal\" not found"
Jul 01 14:39:21.885890 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.885872    2551 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-55.ec2.internal"]
Jul 01 14:39:21.885968 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.885933    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 14:39:21.887317 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.887301    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 14:39:21.887397 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.887331    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 14:39:21.887397 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.887341    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasSufficientPID"
Jul 01 14:39:21.888640 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.888627    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 14:39:21.888751 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.888727    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal"
Jul 01 14:39:21.888789 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.888770    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 14:39:21.889318 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.889296    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 14:39:21.889318 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.889304    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 14:39:21.889401 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.889325    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 14:39:21.889401 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.889339    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasSufficientPID"
Jul 01 14:39:21.889401 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.889327    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 14:39:21.889522 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.889429    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasSufficientPID"
Jul 01 14:39:21.890058 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:21.890039    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-55.ec2.internal\" not found"
Jul 01 14:39:21.890971 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.890948    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-55.ec2.internal"
Jul 01 14:39:21.890971 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.890976    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 14:39:21.891639 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.891623    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 14:39:21.891723 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.891654    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 14:39:21.891723 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.891668    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasSufficientPID"
Jul 01 14:39:21.914880 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.914847    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f26db238ea81eafaa3ded9fde0688757-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal\" (UID: \"f26db238ea81eafaa3ded9fde0688757\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal"
Jul 01 14:39:21.914880 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.914875    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f26db238ea81eafaa3ded9fde0688757-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal\" (UID: \"f26db238ea81eafaa3ded9fde0688757\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal"
Jul 01 14:39:21.915052 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:21.914891    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/814e0bac5e0890277234107c5ef5089c-config\") pod \"kube-apiserver-proxy-ip-10-0-139-55.ec2.internal\" (UID: \"814e0bac5e0890277234107c5ef5089c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-55.ec2.internal"
Jul 01 14:39:21.916294 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:21.916276    2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-55.ec2.internal\" not found" node="ip-10-0-139-55.ec2.internal"
Jul 01 14:39:21.920733 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:21.920712    2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-55.ec2.internal\" not found" node="ip-10-0-139-55.ec2.internal"
Jul 01 14:39:21.990841 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:21.990814    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-55.ec2.internal\" not found"
Jul 01 14:39:22.015061 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:22.015035    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f26db238ea81eafaa3ded9fde0688757-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal\" (UID: \"f26db238ea81eafaa3ded9fde0688757\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal"
Jul 01 14:39:22.015135 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:22.015063    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f26db238ea81eafaa3ded9fde0688757-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal\" (UID: \"f26db238ea81eafaa3ded9fde0688757\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal"
Jul 01 14:39:22.015135 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:22.015084    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/814e0bac5e0890277234107c5ef5089c-config\") pod \"kube-apiserver-proxy-ip-10-0-139-55.ec2.internal\" (UID: \"814e0bac5e0890277234107c5ef5089c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-55.ec2.internal"
Jul 01 14:39:22.015135 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:22.015112    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/814e0bac5e0890277234107c5ef5089c-config\") pod \"kube-apiserver-proxy-ip-10-0-139-55.ec2.internal\" (UID: \"814e0bac5e0890277234107c5ef5089c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-55.ec2.internal"
Jul 01 14:39:22.015227 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:22.015134    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f26db238ea81eafaa3ded9fde0688757-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal\" (UID: \"f26db238ea81eafaa3ded9fde0688757\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal"
Jul 01 14:39:22.015227 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:22.015146    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f26db238ea81eafaa3ded9fde0688757-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal\" (UID: \"f26db238ea81eafaa3ded9fde0688757\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal"
Jul 01 14:39:22.091303 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:22.091254    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-55.ec2.internal\" not found"
Jul 01 14:39:22.191977 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:22.191947    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-55.ec2.internal\" not found"
Jul 01 14:39:22.220506 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:22.220477    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal"
Jul 01 14:39:22.224813 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:22.224792    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-55.ec2.internal"
Jul 01 14:39:22.292642 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:22.292599    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-55.ec2.internal\" not found"
Jul 01 14:39:22.393148 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:22.393110    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-55.ec2.internal\" not found"
Jul 01 14:39:22.493810 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:22.493716    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-55.ec2.internal\" not found"
Jul 01 14:39:22.594288 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:22.594251    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-55.ec2.internal\" not found"
Jul 01 14:39:22.644720 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:22.644685    2551 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 01 14:39:22.645473 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:22.644844    2551 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 01 14:39:22.694738 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:22.694705    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-55.ec2.internal\" not found"
Jul 01 14:39:22.722275 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:22.722253    2551 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 14:39:22.730401 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:22.730361    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf26db238ea81eafaa3ded9fde0688757.slice/crio-82f3f319f29a5082ff45d037de1e0fb8ea430d2f34308738b3aec253427078eb WatchSource:0}: Error finding container 82f3f319f29a5082ff45d037de1e0fb8ea430d2f34308738b3aec253427078eb: Status 404 returned error can't find the container with id 82f3f319f29a5082ff45d037de1e0fb8ea430d2f34308738b3aec253427078eb
Jul 01 14:39:22.730668 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:22.730645    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod814e0bac5e0890277234107c5ef5089c.slice/crio-f8b54169408420b94a8feb8d27556c00a2418b044c9c5dc9508495b9846dfc70 WatchSource:0}: Error finding container f8b54169408420b94a8feb8d27556c00a2418b044c9c5dc9508495b9846dfc70: Status 404 returned error can't find the container with id f8b54169408420b94a8feb8d27556c00a2418b044c9c5dc9508495b9846dfc70
Jul 01 14:39:22.738648 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:22.738629    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 01 14:39:22.746675 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:22.746599    2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-30 14:34:23 +0000 UTC" deadline="2028-03-25 11:33:09.126064264 +0000 UTC"
Jul 01 14:39:22.746675 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:22.746637    2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15188h53m46.379430092s"
Jul 01 14:39:22.789289 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:22.789231    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-55.ec2.internal" event={"ID":"814e0bac5e0890277234107c5ef5089c","Type":"ContainerStarted","Data":"f8b54169408420b94a8feb8d27556c00a2418b044c9c5dc9508495b9846dfc70"}
Jul 01 14:39:22.790210 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:22.790186    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal" event={"ID":"f26db238ea81eafaa3ded9fde0688757","Type":"ContainerStarted","Data":"82f3f319f29a5082ff45d037de1e0fb8ea430d2f34308738b3aec253427078eb"}
Jul 01 14:39:22.795352 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:22.795327    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-55.ec2.internal\" not found"
Jul 01 14:39:22.896091 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:22.896013    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-55.ec2.internal\" not found"
Jul 01 14:39:22.896540 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:22.896526    2551 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 14:39:22.912802 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:22.912774    2551 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal"
Jul 01 14:39:22.924649 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:22.924629    2551 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 01 14:39:22.925680 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:22.925665    2551 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-55.ec2.internal"
Jul 01 14:39:22.937496 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:22.937474    2551 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 01 14:39:24.473691 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:24.473664    2551 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 14:39:25.004096 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.004054    2551 apiserver.go:52] "Watching apiserver"
Jul 01 14:39:25.011747 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.011722    2551 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 01 14:39:25.012191 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.012165    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-km6cc","kube-system/konnectivity-agent-qrcb9","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jrwp","openshift-cluster-node-tuning-operator/tuned-649r6","openshift-dns/node-resolver-rzw6z","openshift-multus/multus-additional-cni-plugins-nhhr7","openshift-multus/network-metrics-daemon-tprn9","openshift-network-operator/iptables-alerter-mgl6l","openshift-ovn-kubernetes/ovnkube-node-qnczw","kube-system/global-pull-secret-syncer-2lqtw","kube-system/kube-apiserver-proxy-ip-10-0-139-55.ec2.internal","openshift-image-registry/node-ca-hn47h","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal","openshift-multus/multus-jxqmr"]
Jul 01 14:39:25.013820 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.013801    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:39:25.013920 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:25.013882    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tprn9" podUID="005968bd-f145-4c81-be95-b3170cafef84"
Jul 01 14:39:25.015292 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.015274    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mgl6l"
Jul 01 14:39:25.016889 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.016865    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jrwp"
Jul 01 14:39:25.017808 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.017790    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 01 14:39:25.018341 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.018314    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 01 14:39:25.018434 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.018350    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2j24w\""
Jul 01 14:39:25.018655 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.018634    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 01 14:39:25.019030 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.019007    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.019121 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.019104    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 01 14:39:25.019352 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.019239    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 01 14:39:25.019652 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.019596    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 01 14:39:25.019738 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.019723    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rmjxk\""
Jul 01 14:39:25.020879 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.020530    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rzw6z"
Jul 01 14:39:25.021317 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.021297    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 01 14:39:25.021429 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.021391    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-j88gg\""
Jul 01 14:39:25.021699 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.021675    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 01 14:39:25.022417 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.022400    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nhhr7"
Jul 01 14:39:25.023332 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.023309    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 01 14:39:25.023446 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.023318    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9b5t4\""
Jul 01 14:39:25.023879 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.023320    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 01 14:39:25.024158 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.024139    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.026386 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.026291    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 01 14:39:25.026484 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.026429    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 01 14:39:25.028139 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.026578    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 01 14:39:25.028139 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.026611    2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 01 14:39:25.028445 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.028418    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:39:25.028915 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:25.028882    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km6cc" podUID="8b8d9348-6896-4fd2-b3c1-8d841219bc43"
Jul 01 14:39:25.029577 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.029556    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 01 14:39:25.029873 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.029855    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 01 14:39:25.029960 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.029921    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 01 14:39:25.030071 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.029957    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bjqkd\""
Jul 01 14:39:25.030071 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.029861    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 01 14:39:25.030071 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.030056    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 01 14:39:25.030465 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.030447    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qrcb9"
Jul 01 14:39:25.030823 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.030807    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 01 14:39:25.031743 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.031713    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 01 14:39:25.031994 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.031892    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:25.031994 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:25.031965    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2lqtw" podUID="99ded651-5fce-49b7-8561-d54826fd2690"
Jul 01 14:39:25.032414 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.032395    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 01 14:39:25.032514 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.032468    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-phlgx\""
Jul 01 14:39:25.032957 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.032934    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 01 14:39:25.033070 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.033042    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 01 14:39:25.033186 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.033167    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-c7rpj\""
Jul 01 14:39:25.033391 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.033373    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.034858 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.034835    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hn47h"
Jul 01 14:39:25.035676 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.035657    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 01 14:39:25.036106 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.036088    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fr5m7\""
Jul 01 14:39:25.036964 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.036943    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnxvg\" (UniqueName: \"kubernetes.io/projected/bc612921-fe17-41c3-93ba-2e1197f6f843-kube-api-access-xnxvg\") pod \"multus-additional-cni-plugins-nhhr7\" (UID: \"bc612921-fe17-41c3-93ba-2e1197f6f843\") " pod="openshift-multus/multus-additional-cni-plugins-nhhr7"
Jul 01 14:39:25.037052 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.036980    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58qfp\" (UniqueName: \"kubernetes.io/projected/4bc04385-5d77-4673-a019-88bb93a250dd-kube-api-access-58qfp\") pod \"node-resolver-rzw6z\" (UID: \"4bc04385-5d77-4673-a019-88bb93a250dd\") " pod="openshift-dns/node-resolver-rzw6z"
Jul 01 14:39:25.037052 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037006    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bc612921-fe17-41c3-93ba-2e1197f6f843-os-release\") pod \"multus-additional-cni-plugins-nhhr7\" (UID: \"bc612921-fe17-41c3-93ba-2e1197f6f843\") " pod="openshift-multus/multus-additional-cni-plugins-nhhr7"
Jul 01 14:39:25.037052 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037041    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-host\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.037210 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037064    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa1455f9-1f95-47e0-9d5f-f73c917257cf-tmp\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.037210 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037088    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-etc-systemd\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.037210 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037116    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6b10226f-d639-4a54-b33f-8b1cb9e35d17-etc-selinux\") pod \"aws-ebs-csi-driver-node-4jrwp\" (UID: \"6b10226f-d639-4a54-b33f-8b1cb9e35d17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jrwp"
Jul 01 14:39:25.037210 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037139    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-host-slash\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.037210 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037150    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 01 14:39:25.037210 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037161    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-host-cni-netd\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.037210 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037188    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-etc-sysctl-d\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.037210 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037210    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-lib-modules\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.037586 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037231    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fa1455f9-1f95-47e0-9d5f-f73c917257cf-etc-tuned\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.037586 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037242    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 01 14:39:25.037586 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037254    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-sys\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.037586 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037277    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bc612921-fe17-41c3-93ba-2e1197f6f843-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nhhr7\" (UID: \"bc612921-fe17-41c3-93ba-2e1197f6f843\") " pod="openshift-multus/multus-additional-cni-plugins-nhhr7"
Jul 01 14:39:25.037586 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037302    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/768a96c0-935c-4629-b015-1b85b0ac3199-ovn-node-metrics-cert\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.037586 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037324    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-etc-kubernetes\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.037586 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037339    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 01 14:39:25.037586 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037348    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsszh\" (UniqueName: \"kubernetes.io/projected/8b8d9348-6896-4fd2-b3c1-8d841219bc43-kube-api-access-tsszh\") pod \"network-check-target-km6cc\" (UID: \"8b8d9348-6896-4fd2-b3c1-8d841219bc43\") " pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:39:25.037586 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037371    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-run-systemd\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.037586 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037395    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-node-log\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.037586 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037419    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-host-cni-bin\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.037586 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037441    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.037586 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037481    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/768a96c0-935c-4629-b015-1b85b0ac3199-env-overrides\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.037586 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037488    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-m9tqs\""
Jul 01 14:39:25.037586 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037504    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6b10226f-d639-4a54-b33f-8b1cb9e35d17-device-dir\") pod \"aws-ebs-csi-driver-node-4jrwp\" (UID: \"6b10226f-d639-4a54-b33f-8b1cb9e35d17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jrwp"
Jul 01 14:39:25.037586 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037527    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bc04385-5d77-4673-a019-88bb93a250dd-tmp-dir\") pod \"node-resolver-rzw6z\" (UID: \"4bc04385-5d77-4673-a019-88bb93a250dd\") " pod="openshift-dns/node-resolver-rzw6z"
Jul 01 14:39:25.037586 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037553    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bc612921-fe17-41c3-93ba-2e1197f6f843-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nhhr7\" (UID: \"bc612921-fe17-41c3-93ba-2e1197f6f843\") " pod="openshift-multus/multus-additional-cni-plugins-nhhr7"
Jul 01 14:39:25.037586 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037584    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-host-kubelet\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.038473 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037608    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-host-run-ovn-kubernetes\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.038473 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037716    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-var-lib-kubelet\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.038473 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037756    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/005968bd-f145-4c81-be95-b3170cafef84-metrics-certs\") pod \"network-metrics-daemon-tprn9\" (UID: \"005968bd-f145-4c81-be95-b3170cafef84\") " pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:39:25.038473 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037808    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bc612921-fe17-41c3-93ba-2e1197f6f843-cni-binary-copy\") pod \"multus-additional-cni-plugins-nhhr7\" (UID: \"bc612921-fe17-41c3-93ba-2e1197f6f843\") " pod="openshift-multus/multus-additional-cni-plugins-nhhr7"
Jul 01 14:39:25.038473 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037860    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-etc-sysconfig\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.038473 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037890    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6b10226f-d639-4a54-b33f-8b1cb9e35d17-registration-dir\") pod \"aws-ebs-csi-driver-node-4jrwp\" (UID: \"6b10226f-d639-4a54-b33f-8b1cb9e35d17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jrwp"
Jul 01 14:39:25.038473 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037919    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bc612921-fe17-41c3-93ba-2e1197f6f843-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nhhr7\" (UID: \"bc612921-fe17-41c3-93ba-2e1197f6f843\") " pod="openshift-multus/multus-additional-cni-plugins-nhhr7"
Jul 01 14:39:25.038473 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.037986    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-var-lib-openvswitch\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.038473 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.038034    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-etc-openvswitch\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.038473 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.038083    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-run-ovn\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.038473 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.038118    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsg8x\" (UniqueName: \"kubernetes.io/projected/fa1455f9-1f95-47e0-9d5f-f73c917257cf-kube-api-access-rsg8x\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.038473 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.038172    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3ea52ec7-825f-4d91-9215-17d64f0b497d-iptables-alerter-script\") pod \"iptables-alerter-mgl6l\" (UID: \"3ea52ec7-825f-4d91-9215-17d64f0b497d\") " pod="openshift-network-operator/iptables-alerter-mgl6l"
Jul 01 14:39:25.038473 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.038198    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-host-run-netns\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.038473 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.038219    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-log-socket\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.038473 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.038239    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bd8r8\" (UniqueName: \"kubernetes.io/projected/768a96c0-935c-4629-b015-1b85b0ac3199-kube-api-access-bd8r8\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.038473 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.038261    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-run\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.039214 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.038286    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3ea52ec7-825f-4d91-9215-17d64f0b497d-host-slash\") pod \"iptables-alerter-mgl6l\" (UID: \"3ea52ec7-825f-4d91-9215-17d64f0b497d\") " pod="openshift-network-operator/iptables-alerter-mgl6l"
Jul 01 14:39:25.039214 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.038308    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6b10226f-d639-4a54-b33f-8b1cb9e35d17-sys-fs\") pod \"aws-ebs-csi-driver-node-4jrwp\" (UID: \"6b10226f-d639-4a54-b33f-8b1cb9e35d17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jrwp"
Jul 01 14:39:25.039214 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.038367    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bc612921-fe17-41c3-93ba-2e1197f6f843-cnibin\") pod \"multus-additional-cni-plugins-nhhr7\" (UID: \"bc612921-fe17-41c3-93ba-2e1197f6f843\") " pod="openshift-multus/multus-additional-cni-plugins-nhhr7"
Jul 01 14:39:25.039214 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.038412    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-run-openvswitch\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.039214 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.038436    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-etc-sysctl-conf\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.039214 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.038475    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6b10226f-d639-4a54-b33f-8b1cb9e35d17-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4jrwp\" (UID: \"6b10226f-d639-4a54-b33f-8b1cb9e35d17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jrwp"
Jul 01 14:39:25.039214 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.038599    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqlcp\" (UniqueName: \"kubernetes.io/projected/6b10226f-d639-4a54-b33f-8b1cb9e35d17-kube-api-access-bqlcp\") pod \"aws-ebs-csi-driver-node-4jrwp\" (UID: \"6b10226f-d639-4a54-b33f-8b1cb9e35d17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jrwp"
Jul 01 14:39:25.039214 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.038654    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc612921-fe17-41c3-93ba-2e1197f6f843-system-cni-dir\") pod \"multus-additional-cni-plugins-nhhr7\" (UID: \"bc612921-fe17-41c3-93ba-2e1197f6f843\") " pod="openshift-multus/multus-additional-cni-plugins-nhhr7"
Jul 01 14:39:25.039214 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.038677    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-etc-modprobe-d\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.039214 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.038700    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-systemd-units\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.039214 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.038746    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/768a96c0-935c-4629-b015-1b85b0ac3199-ovnkube-config\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.039214 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.038768    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/768a96c0-935c-4629-b015-1b85b0ac3199-ovnkube-script-lib\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.039214 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.038791    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6b10226f-d639-4a54-b33f-8b1cb9e35d17-socket-dir\") pod \"aws-ebs-csi-driver-node-4jrwp\" (UID: \"6b10226f-d639-4a54-b33f-8b1cb9e35d17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jrwp"
Jul 01 14:39:25.039214 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.038871    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnmsm\" (UniqueName: \"kubernetes.io/projected/005968bd-f145-4c81-be95-b3170cafef84-kube-api-access-mnmsm\") pod \"network-metrics-daemon-tprn9\" (UID: \"005968bd-f145-4c81-be95-b3170cafef84\") " pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:39:25.039214 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.038897    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5c2dq\" (UniqueName: \"kubernetes.io/projected/3ea52ec7-825f-4d91-9215-17d64f0b497d-kube-api-access-5c2dq\") pod \"iptables-alerter-mgl6l\" (UID: \"3ea52ec7-825f-4d91-9215-17d64f0b497d\") " pod="openshift-network-operator/iptables-alerter-mgl6l"
Jul 01 14:39:25.039214 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.038919    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4bc04385-5d77-4673-a019-88bb93a250dd-hosts-file\") pod \"node-resolver-rzw6z\" (UID: \"4bc04385-5d77-4673-a019-88bb93a250dd\") " pod="openshift-dns/node-resolver-rzw6z"
Jul 01 14:39:25.040428 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.040409    2551 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 01 14:39:25.066582 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.066554    2551 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2rpxx"
Jul 01 14:39:25.074776 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.074755    2551 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2rpxx"
Jul 01 14:39:25.126397 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.126363    2551 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 01 14:39:25.139576 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.139539    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-run\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.139576 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.139586    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0710008c-6eee-4fd1-aac9-81df1341171f-konnectivity-ca\") pod \"konnectivity-agent-qrcb9\" (UID: \"0710008c-6eee-4fd1-aac9-81df1341171f\") " pod="kube-system/konnectivity-agent-qrcb9"
Jul 01 14:39:25.139829 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.139608    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/99ded651-5fce-49b7-8561-d54826fd2690-kubelet-config\") pod \"global-pull-secret-syncer-2lqtw\" (UID: \"99ded651-5fce-49b7-8561-d54826fd2690\") " pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:25.139829 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.139641    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-host-run-k8s-cni-cncf-io\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.139829 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.139662    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3ea52ec7-825f-4d91-9215-17d64f0b497d-host-slash\") pod \"iptables-alerter-mgl6l\" (UID: \"3ea52ec7-825f-4d91-9215-17d64f0b497d\") " pod="openshift-network-operator/iptables-alerter-mgl6l"
Jul 01 14:39:25.139829 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.139686    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6b10226f-d639-4a54-b33f-8b1cb9e35d17-sys-fs\") pod \"aws-ebs-csi-driver-node-4jrwp\" (UID: \"6b10226f-d639-4a54-b33f-8b1cb9e35d17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jrwp"
Jul 01 14:39:25.139829 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.139702    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bc612921-fe17-41c3-93ba-2e1197f6f843-cnibin\") pod \"multus-additional-cni-plugins-nhhr7\" (UID: \"bc612921-fe17-41c3-93ba-2e1197f6f843\") " pod="openshift-multus/multus-additional-cni-plugins-nhhr7"
Jul 01 14:39:25.139829 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.139701    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-run\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.139829 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.139715    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3ea52ec7-825f-4d91-9215-17d64f0b497d-host-slash\") pod \"iptables-alerter-mgl6l\" (UID: \"3ea52ec7-825f-4d91-9215-17d64f0b497d\") " pod="openshift-network-operator/iptables-alerter-mgl6l"
Jul 01 14:39:25.139829 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.139725    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-run-openvswitch\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.139829 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.139768    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-etc-sysctl-conf\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.139829 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.139779    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bc612921-fe17-41c3-93ba-2e1197f6f843-cnibin\") pod \"multus-additional-cni-plugins-nhhr7\" (UID: \"bc612921-fe17-41c3-93ba-2e1197f6f843\") " pod="openshift-multus/multus-additional-cni-plugins-nhhr7"
Jul 01 14:39:25.139829 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.139791    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6b10226f-d639-4a54-b33f-8b1cb9e35d17-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4jrwp\" (UID: \"6b10226f-d639-4a54-b33f-8b1cb9e35d17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jrwp"
Jul 01 14:39:25.139829 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.139808    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-run-openvswitch\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.139829 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.139815    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqlcp\" (UniqueName: \"kubernetes.io/projected/6b10226f-d639-4a54-b33f-8b1cb9e35d17-kube-api-access-bqlcp\") pod \"aws-ebs-csi-driver-node-4jrwp\" (UID: \"6b10226f-d639-4a54-b33f-8b1cb9e35d17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jrwp"
Jul 01 14:39:25.139829 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.139812    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6b10226f-d639-4a54-b33f-8b1cb9e35d17-sys-fs\") pod \"aws-ebs-csi-driver-node-4jrwp\" (UID: \"6b10226f-d639-4a54-b33f-8b1cb9e35d17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jrwp"
Jul 01 14:39:25.140452 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.139871    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc612921-fe17-41c3-93ba-2e1197f6f843-system-cni-dir\") pod \"multus-additional-cni-plugins-nhhr7\" (UID: \"bc612921-fe17-41c3-93ba-2e1197f6f843\") " pod="openshift-multus/multus-additional-cni-plugins-nhhr7"
Jul 01 14:39:25.140452 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.139887    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6b10226f-d639-4a54-b33f-8b1cb9e35d17-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4jrwp\" (UID: \"6b10226f-d639-4a54-b33f-8b1cb9e35d17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jrwp"
Jul 01 14:39:25.140452 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.139927    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-etc-sysctl-conf\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.140452 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.139931    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-etc-modprobe-d\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.140452 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.139968    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc612921-fe17-41c3-93ba-2e1197f6f843-system-cni-dir\") pod \"multus-additional-cni-plugins-nhhr7\" (UID: \"bc612921-fe17-41c3-93ba-2e1197f6f843\") " pod="openshift-multus/multus-additional-cni-plugins-nhhr7"
Jul 01 14:39:25.140452 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140033    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/49ff50e4-fbc4-4623-a5c5-458c628adef3-host\") pod \"node-ca-hn47h\" (UID: \"49ff50e4-fbc4-4623-a5c5-458c628adef3\") " pod="openshift-image-registry/node-ca-hn47h"
Jul 01 14:39:25.140452 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140065    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-systemd-units\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.140452 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140098    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-etc-modprobe-d\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.140452 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140107    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-systemd-units\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.140452 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140110    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/768a96c0-935c-4629-b015-1b85b0ac3199-ovnkube-config\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.140452 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140136    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/768a96c0-935c-4629-b015-1b85b0ac3199-ovnkube-script-lib\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.140452 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140162    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-cnibin\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.140452 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140187    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6b10226f-d639-4a54-b33f-8b1cb9e35d17-socket-dir\") pod \"aws-ebs-csi-driver-node-4jrwp\" (UID: \"6b10226f-d639-4a54-b33f-8b1cb9e35d17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jrwp"
Jul 01 14:39:25.140452 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140206    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0710008c-6eee-4fd1-aac9-81df1341171f-agent-certs\") pod \"konnectivity-agent-qrcb9\" (UID: \"0710008c-6eee-4fd1-aac9-81df1341171f\") " pod="kube-system/konnectivity-agent-qrcb9"
Jul 01 14:39:25.140452 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140222    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-os-release\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.140452 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140244    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-cni-binary-copy\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.140452 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140269    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-multus-daemon-config\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.141239 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140307    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57mfr\" (UniqueName: \"kubernetes.io/projected/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-kube-api-access-57mfr\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.141239 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140341    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6b10226f-d639-4a54-b33f-8b1cb9e35d17-socket-dir\") pod \"aws-ebs-csi-driver-node-4jrwp\" (UID: \"6b10226f-d639-4a54-b33f-8b1cb9e35d17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jrwp"
Jul 01 14:39:25.141239 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140361    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnmsm\" (UniqueName: \"kubernetes.io/projected/005968bd-f145-4c81-be95-b3170cafef84-kube-api-access-mnmsm\") pod \"network-metrics-daemon-tprn9\" (UID: \"005968bd-f145-4c81-be95-b3170cafef84\") " pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:39:25.141239 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140402    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c2dq\" (UniqueName: \"kubernetes.io/projected/3ea52ec7-825f-4d91-9215-17d64f0b497d-kube-api-access-5c2dq\") pod \"iptables-alerter-mgl6l\" (UID: \"3ea52ec7-825f-4d91-9215-17d64f0b497d\") " pod="openshift-network-operator/iptables-alerter-mgl6l"
Jul 01 14:39:25.141239 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140429    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4bc04385-5d77-4673-a019-88bb93a250dd-hosts-file\") pod \"node-resolver-rzw6z\" (UID: \"4bc04385-5d77-4673-a019-88bb93a250dd\") " pod="openshift-dns/node-resolver-rzw6z"
Jul 01 14:39:25.141239 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140453    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-host-var-lib-cni-bin\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.141239 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140476    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-etc-kubernetes\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.141239 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140501    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnxvg\" (UniqueName: \"kubernetes.io/projected/bc612921-fe17-41c3-93ba-2e1197f6f843-kube-api-access-xnxvg\") pod \"multus-additional-cni-plugins-nhhr7\" (UID: \"bc612921-fe17-41c3-93ba-2e1197f6f843\") " pod="openshift-multus/multus-additional-cni-plugins-nhhr7"
Jul 01 14:39:25.141239 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140525    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/99ded651-5fce-49b7-8561-d54826fd2690-original-pull-secret\") pod \"global-pull-secret-syncer-2lqtw\" (UID: \"99ded651-5fce-49b7-8561-d54826fd2690\") " pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:25.141239 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140531    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4bc04385-5d77-4673-a019-88bb93a250dd-hosts-file\") pod \"node-resolver-rzw6z\" (UID: \"4bc04385-5d77-4673-a019-88bb93a250dd\") " pod="openshift-dns/node-resolver-rzw6z"
Jul 01 14:39:25.141239 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140549    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-multus-socket-dir-parent\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.141239 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140582    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58qfp\" (UniqueName: \"kubernetes.io/projected/4bc04385-5d77-4673-a019-88bb93a250dd-kube-api-access-58qfp\") pod \"node-resolver-rzw6z\" (UID: \"4bc04385-5d77-4673-a019-88bb93a250dd\") " pod="openshift-dns/node-resolver-rzw6z"
Jul 01 14:39:25.141239 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140657    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bc612921-fe17-41c3-93ba-2e1197f6f843-os-release\") pod \"multus-additional-cni-plugins-nhhr7\" (UID: \"bc612921-fe17-41c3-93ba-2e1197f6f843\") " pod="openshift-multus/multus-additional-cni-plugins-nhhr7"
Jul 01 14:39:25.141239 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140684    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-host\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.141239 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140709    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa1455f9-1f95-47e0-9d5f-f73c917257cf-tmp\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.141239 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140737    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-etc-systemd\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.141239 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140744    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bc612921-fe17-41c3-93ba-2e1197f6f843-os-release\") pod \"multus-additional-cni-plugins-nhhr7\" (UID: \"bc612921-fe17-41c3-93ba-2e1197f6f843\") " pod="openshift-multus/multus-additional-cni-plugins-nhhr7"
Jul 01 14:39:25.142034 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140771    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-host\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.142034 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140788    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-host-var-lib-kubelet\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.142034 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140836    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6b10226f-d639-4a54-b33f-8b1cb9e35d17-etc-selinux\") pod \"aws-ebs-csi-driver-node-4jrwp\" (UID: \"6b10226f-d639-4a54-b33f-8b1cb9e35d17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jrwp"
Jul 01 14:39:25.142034 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140846    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/768a96c0-935c-4629-b015-1b85b0ac3199-ovnkube-script-lib\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.142034 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140841    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-etc-systemd\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.142034 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140868    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-host-slash\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.142034 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140901    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/768a96c0-935c-4629-b015-1b85b0ac3199-ovnkube-config\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.142034 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140911    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-host-slash\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.142034 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140904    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-host-cni-netd\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.142034 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140950    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-host-cni-netd\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.142034 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140940    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6b10226f-d639-4a54-b33f-8b1cb9e35d17-etc-selinux\") pod \"aws-ebs-csi-driver-node-4jrwp\" (UID: \"6b10226f-d639-4a54-b33f-8b1cb9e35d17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jrwp"
Jul 01 14:39:25.142034 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140953    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-etc-sysctl-d\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.142034 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.140997    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-lib-modules\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.142034 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141021    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fa1455f9-1f95-47e0-9d5f-f73c917257cf-etc-tuned\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.142034 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141045    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-sys\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.142034 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141071    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bc612921-fe17-41c3-93ba-2e1197f6f843-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nhhr7\" (UID: \"bc612921-fe17-41c3-93ba-2e1197f6f843\") " pod="openshift-multus/multus-additional-cni-plugins-nhhr7"
Jul 01 14:39:25.142034 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141082    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-etc-sysctl-d\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.142817 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141098    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/768a96c0-935c-4629-b015-1b85b0ac3199-ovn-node-metrics-cert\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.142817 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141114    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-sys\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.142817 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141149    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-etc-kubernetes\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.142817 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141151    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-lib-modules\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.142817 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141157    2551 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 01 14:39:25.142817 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141176    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsszh\" (UniqueName: \"kubernetes.io/projected/8b8d9348-6896-4fd2-b3c1-8d841219bc43-kube-api-access-tsszh\") pod \"network-check-target-km6cc\" (UID: \"8b8d9348-6896-4fd2-b3c1-8d841219bc43\") " pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:39:25.142817 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141214    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-etc-kubernetes\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.142817 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141218    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-host-run-multus-certs\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.142817 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141247    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-run-systemd\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.142817 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141263    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bc612921-fe17-41c3-93ba-2e1197f6f843-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nhhr7\" (UID: \"bc612921-fe17-41c3-93ba-2e1197f6f843\") " pod="openshift-multus/multus-additional-cni-plugins-nhhr7"
Jul 01 14:39:25.142817 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141276    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-run-systemd\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.142817 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141270    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-node-log\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.142817 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141304    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-node-log\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.142817 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141319    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-host-cni-bin\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.142817 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141345    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.142817 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141388    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/768a96c0-935c-4629-b015-1b85b0ac3199-env-overrides\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.142817 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141423    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.143600 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141427    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-multus-conf-dir\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.143600 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141426    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-host-cni-bin\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.143600 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141544    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6b10226f-d639-4a54-b33f-8b1cb9e35d17-device-dir\") pod \"aws-ebs-csi-driver-node-4jrwp\" (UID: \"6b10226f-d639-4a54-b33f-8b1cb9e35d17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jrwp"
Jul 01 14:39:25.143600 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141600    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bc04385-5d77-4673-a019-88bb93a250dd-tmp-dir\") pod \"node-resolver-rzw6z\" (UID: \"4bc04385-5d77-4673-a019-88bb93a250dd\") " pod="openshift-dns/node-resolver-rzw6z"
Jul 01 14:39:25.143600 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141643    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bc612921-fe17-41c3-93ba-2e1197f6f843-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nhhr7\" (UID: \"bc612921-fe17-41c3-93ba-2e1197f6f843\") " pod="openshift-multus/multus-additional-cni-plugins-nhhr7"
Jul 01 14:39:25.143600 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141667    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-host-kubelet\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.143600 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141686    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6b10226f-d639-4a54-b33f-8b1cb9e35d17-device-dir\") pod \"aws-ebs-csi-driver-node-4jrwp\" (UID: \"6b10226f-d639-4a54-b33f-8b1cb9e35d17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jrwp"
Jul 01 14:39:25.143600 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141690    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-host-run-ovn-kubernetes\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.143600 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141725    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-host-run-ovn-kubernetes\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.143600 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141770    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-hostroot\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.143600 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141800    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/768a96c0-935c-4629-b015-1b85b0ac3199-env-overrides\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.143600 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141791    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/49ff50e4-fbc4-4623-a5c5-458c628adef3-serviceca\") pod \"node-ca-hn47h\" (UID: \"49ff50e4-fbc4-4623-a5c5-458c628adef3\") " pod="openshift-image-registry/node-ca-hn47h"
Jul 01 14:39:25.143600 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141844    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-host-kubelet\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.143600 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141910    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-var-lib-kubelet\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.143600 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141932    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/99ded651-5fce-49b7-8561-d54826fd2690-dbus\") pod \"global-pull-secret-syncer-2lqtw\" (UID: \"99ded651-5fce-49b7-8561-d54826fd2690\") " pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:25.143600 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.141961    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-var-lib-kubelet\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.143600 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.142009    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-system-cni-dir\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.144463 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.142041    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ww49\" (UniqueName: \"kubernetes.io/projected/49ff50e4-fbc4-4623-a5c5-458c628adef3-kube-api-access-5ww49\") pod \"node-ca-hn47h\" (UID: \"49ff50e4-fbc4-4623-a5c5-458c628adef3\") " pod="openshift-image-registry/node-ca-hn47h"
Jul 01 14:39:25.144463 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.142080    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/005968bd-f145-4c81-be95-b3170cafef84-metrics-certs\") pod \"network-metrics-daemon-tprn9\" (UID: \"005968bd-f145-4c81-be95-b3170cafef84\") " pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:39:25.144463 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.142125    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bc612921-fe17-41c3-93ba-2e1197f6f843-cni-binary-copy\") pod \"multus-additional-cni-plugins-nhhr7\" (UID: \"bc612921-fe17-41c3-93ba-2e1197f6f843\") " pod="openshift-multus/multus-additional-cni-plugins-nhhr7"
Jul 01 14:39:25.144463 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:25.142224    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 14:39:25.144463 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:25.142290    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/005968bd-f145-4c81-be95-b3170cafef84-metrics-certs podName:005968bd-f145-4c81-be95-b3170cafef84 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:25.642269311 +0000 UTC m=+2.970841610 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/005968bd-f145-4c81-be95-b3170cafef84-metrics-certs") pod "network-metrics-daemon-tprn9" (UID: "005968bd-f145-4c81-be95-b3170cafef84") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 14:39:25.144463 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.142325    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bc612921-fe17-41c3-93ba-2e1197f6f843-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nhhr7\" (UID: \"bc612921-fe17-41c3-93ba-2e1197f6f843\") " pod="openshift-multus/multus-additional-cni-plugins-nhhr7"
Jul 01 14:39:25.144463 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.142337    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bc04385-5d77-4673-a019-88bb93a250dd-tmp-dir\") pod \"node-resolver-rzw6z\" (UID: \"4bc04385-5d77-4673-a019-88bb93a250dd\") " pod="openshift-dns/node-resolver-rzw6z"
Jul 01 14:39:25.144463 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.142427    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-etc-sysconfig\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.144463 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.142459    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-multus-cni-dir\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.144463 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.142488    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-host-var-lib-cni-multus\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.144463 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.142534    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fa1455f9-1f95-47e0-9d5f-f73c917257cf-etc-sysconfig\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.144463 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.142571    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6b10226f-d639-4a54-b33f-8b1cb9e35d17-registration-dir\") pod \"aws-ebs-csi-driver-node-4jrwp\" (UID: \"6b10226f-d639-4a54-b33f-8b1cb9e35d17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jrwp"
Jul 01 14:39:25.144463 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.142644    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bc612921-fe17-41c3-93ba-2e1197f6f843-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nhhr7\" (UID: \"bc612921-fe17-41c3-93ba-2e1197f6f843\") " pod="openshift-multus/multus-additional-cni-plugins-nhhr7"
Jul 01 14:39:25.144463 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.142681    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6b10226f-d639-4a54-b33f-8b1cb9e35d17-registration-dir\") pod \"aws-ebs-csi-driver-node-4jrwp\" (UID: \"6b10226f-d639-4a54-b33f-8b1cb9e35d17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jrwp"
Jul 01 14:39:25.144463 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.142699    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-var-lib-openvswitch\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.144463 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.142729    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-etc-openvswitch\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.144463 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.142756    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-run-ovn\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.145272 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.142811    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsg8x\" (UniqueName: \"kubernetes.io/projected/fa1455f9-1f95-47e0-9d5f-f73c917257cf-kube-api-access-rsg8x\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.145272 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.142815    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-var-lib-openvswitch\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.145272 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.142832    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-etc-openvswitch\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.145272 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.142776    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bc612921-fe17-41c3-93ba-2e1197f6f843-cni-binary-copy\") pod \"multus-additional-cni-plugins-nhhr7\" (UID: \"bc612921-fe17-41c3-93ba-2e1197f6f843\") " pod="openshift-multus/multus-additional-cni-plugins-nhhr7"
Jul 01 14:39:25.145272 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.142837    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-host-run-netns\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.145272 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.142885    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-run-ovn\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.145272 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.142890    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3ea52ec7-825f-4d91-9215-17d64f0b497d-iptables-alerter-script\") pod \"iptables-alerter-mgl6l\" (UID: \"3ea52ec7-825f-4d91-9215-17d64f0b497d\") " pod="openshift-network-operator/iptables-alerter-mgl6l"
Jul 01 14:39:25.145272 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.142916    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-host-run-netns\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.145272 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.142966    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-log-socket\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.145272 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.142995    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bd8r8\" (UniqueName: \"kubernetes.io/projected/768a96c0-935c-4629-b015-1b85b0ac3199-kube-api-access-bd8r8\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.145272 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.143111    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-host-run-netns\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.145272 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.143147    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bc612921-fe17-41c3-93ba-2e1197f6f843-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nhhr7\" (UID: \"bc612921-fe17-41c3-93ba-2e1197f6f843\") " pod="openshift-multus/multus-additional-cni-plugins-nhhr7"
Jul 01 14:39:25.145272 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.143166    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/768a96c0-935c-4629-b015-1b85b0ac3199-log-socket\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.145272 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.143377    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3ea52ec7-825f-4d91-9215-17d64f0b497d-iptables-alerter-script\") pod \"iptables-alerter-mgl6l\" (UID: \"3ea52ec7-825f-4d91-9215-17d64f0b497d\") " pod="openshift-network-operator/iptables-alerter-mgl6l"
Jul 01 14:39:25.145272 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.144753    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa1455f9-1f95-47e0-9d5f-f73c917257cf-tmp\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.145272 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.144788    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/768a96c0-935c-4629-b015-1b85b0ac3199-ovn-node-metrics-cert\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.145272 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.144922    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fa1455f9-1f95-47e0-9d5f-f73c917257cf-etc-tuned\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.150154 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:25.150125    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 14:39:25.150275 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:25.150159    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 14:39:25.150275 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:25.150171    2551 projected.go:196] Error preparing data for projected volume kube-api-access-tsszh for pod openshift-network-diagnostics/network-check-target-km6cc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 14:39:25.150275 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:25.150258    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b8d9348-6896-4fd2-b3c1-8d841219bc43-kube-api-access-tsszh podName:8b8d9348-6896-4fd2-b3c1-8d841219bc43 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:25.650241562 +0000 UTC m=+2.978813859 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tsszh" (UniqueName: "kubernetes.io/projected/8b8d9348-6896-4fd2-b3c1-8d841219bc43-kube-api-access-tsszh") pod "network-check-target-km6cc" (UID: "8b8d9348-6896-4fd2-b3c1-8d841219bc43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 14:39:25.152744 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.152702    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnmsm\" (UniqueName: \"kubernetes.io/projected/005968bd-f145-4c81-be95-b3170cafef84-kube-api-access-mnmsm\") pod \"network-metrics-daemon-tprn9\" (UID: \"005968bd-f145-4c81-be95-b3170cafef84\") " pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:39:25.152923 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.152898    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5c2dq\" (UniqueName: \"kubernetes.io/projected/3ea52ec7-825f-4d91-9215-17d64f0b497d-kube-api-access-5c2dq\") pod \"iptables-alerter-mgl6l\" (UID: \"3ea52ec7-825f-4d91-9215-17d64f0b497d\") " pod="openshift-network-operator/iptables-alerter-mgl6l"
Jul 01 14:39:25.153510 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.153463    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58qfp\" (UniqueName: \"kubernetes.io/projected/4bc04385-5d77-4673-a019-88bb93a250dd-kube-api-access-58qfp\") pod \"node-resolver-rzw6z\" (UID: \"4bc04385-5d77-4673-a019-88bb93a250dd\") " pod="openshift-dns/node-resolver-rzw6z"
Jul 01 14:39:25.153510 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.153464    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqlcp\" (UniqueName: \"kubernetes.io/projected/6b10226f-d639-4a54-b33f-8b1cb9e35d17-kube-api-access-bqlcp\") pod \"aws-ebs-csi-driver-node-4jrwp\" (UID: \"6b10226f-d639-4a54-b33f-8b1cb9e35d17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jrwp"
Jul 01 14:39:25.153897 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.153880    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnxvg\" (UniqueName: \"kubernetes.io/projected/bc612921-fe17-41c3-93ba-2e1197f6f843-kube-api-access-xnxvg\") pod \"multus-additional-cni-plugins-nhhr7\" (UID: \"bc612921-fe17-41c3-93ba-2e1197f6f843\") " pod="openshift-multus/multus-additional-cni-plugins-nhhr7"
Jul 01 14:39:25.156494 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.156472    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsg8x\" (UniqueName: \"kubernetes.io/projected/fa1455f9-1f95-47e0-9d5f-f73c917257cf-kube-api-access-rsg8x\") pod \"tuned-649r6\" (UID: \"fa1455f9-1f95-47e0-9d5f-f73c917257cf\") " pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.156875 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.156854    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bd8r8\" (UniqueName: \"kubernetes.io/projected/768a96c0-935c-4629-b015-1b85b0ac3199-kube-api-access-bd8r8\") pod \"ovnkube-node-qnczw\" (UID: \"768a96c0-935c-4629-b015-1b85b0ac3199\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.243321 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.243284    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-host-run-netns\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.243486 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.243331    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0710008c-6eee-4fd1-aac9-81df1341171f-konnectivity-ca\") pod \"konnectivity-agent-qrcb9\" (UID: \"0710008c-6eee-4fd1-aac9-81df1341171f\") " pod="kube-system/konnectivity-agent-qrcb9"
Jul 01 14:39:25.243486 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.243351    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/99ded651-5fce-49b7-8561-d54826fd2690-kubelet-config\") pod \"global-pull-secret-syncer-2lqtw\" (UID: \"99ded651-5fce-49b7-8561-d54826fd2690\") " pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:25.243486 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.243368    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-host-run-k8s-cni-cncf-io\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.243486 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.243396    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/49ff50e4-fbc4-4623-a5c5-458c628adef3-host\") pod \"node-ca-hn47h\" (UID: \"49ff50e4-fbc4-4623-a5c5-458c628adef3\") " pod="openshift-image-registry/node-ca-hn47h"
Jul 01 14:39:25.243486 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.243410    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-host-run-netns\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.243486 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.243434    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/99ded651-5fce-49b7-8561-d54826fd2690-kubelet-config\") pod \"global-pull-secret-syncer-2lqtw\" (UID: \"99ded651-5fce-49b7-8561-d54826fd2690\") " pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:25.243486 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.243442    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/49ff50e4-fbc4-4623-a5c5-458c628adef3-host\") pod \"node-ca-hn47h\" (UID: \"49ff50e4-fbc4-4623-a5c5-458c628adef3\") " pod="openshift-image-registry/node-ca-hn47h"
Jul 01 14:39:25.243486 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.243461    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-host-run-k8s-cni-cncf-io\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.243918 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.243501    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-cnibin\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.243918 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.243535    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0710008c-6eee-4fd1-aac9-81df1341171f-agent-certs\") pod \"konnectivity-agent-qrcb9\" (UID: \"0710008c-6eee-4fd1-aac9-81df1341171f\") " pod="kube-system/konnectivity-agent-qrcb9"
Jul 01 14:39:25.243918 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.243556    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-os-release\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.243918 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.243560    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-cnibin\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.243918 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.243580    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-cni-binary-copy\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.243918 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.243668    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-os-release\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.243918 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.243707    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-multus-daemon-config\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.243918 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.243737    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57mfr\" (UniqueName: \"kubernetes.io/projected/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-kube-api-access-57mfr\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.243918 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.243766    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-host-var-lib-cni-bin\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.243918 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.243788    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-etc-kubernetes\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.243918 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.243817    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/99ded651-5fce-49b7-8561-d54826fd2690-original-pull-secret\") pod \"global-pull-secret-syncer-2lqtw\" (UID: \"99ded651-5fce-49b7-8561-d54826fd2690\") " pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:25.243918 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.243842    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-multus-socket-dir-parent\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.243918 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.243866    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0710008c-6eee-4fd1-aac9-81df1341171f-konnectivity-ca\") pod \"konnectivity-agent-qrcb9\" (UID: \"0710008c-6eee-4fd1-aac9-81df1341171f\") " pod="kube-system/konnectivity-agent-qrcb9"
Jul 01 14:39:25.243918 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.243880    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-host-var-lib-kubelet\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.243918 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.243924    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-host-var-lib-kubelet\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.244593 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.243942    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-host-run-multus-certs\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.244593 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.243998    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-multus-conf-dir\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.244593 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.244058    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-hostroot\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.244593 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.244082    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/49ff50e4-fbc4-4623-a5c5-458c628adef3-serviceca\") pod \"node-ca-hn47h\" (UID: \"49ff50e4-fbc4-4623-a5c5-458c628adef3\") " pod="openshift-image-registry/node-ca-hn47h"
Jul 01 14:39:25.244593 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.244098    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-cni-binary-copy\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.244593 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.244115    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-etc-kubernetes\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.244593 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.244128    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/99ded651-5fce-49b7-8561-d54826fd2690-dbus\") pod \"global-pull-secret-syncer-2lqtw\" (UID: \"99ded651-5fce-49b7-8561-d54826fd2690\") " pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:25.244593 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.244191    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-multus-socket-dir-parent\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.244593 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.244200    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-system-cni-dir\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.244593 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:25.244208    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 14:39:25.244593 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.244226    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ww49\" (UniqueName: \"kubernetes.io/projected/49ff50e4-fbc4-4623-a5c5-458c628adef3-kube-api-access-5ww49\") pod \"node-ca-hn47h\" (UID: \"49ff50e4-fbc4-4623-a5c5-458c628adef3\") " pod="openshift-image-registry/node-ca-hn47h"
Jul 01 14:39:25.244593 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.244234    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-hostroot\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.244593 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:25.244280    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99ded651-5fce-49b7-8561-d54826fd2690-original-pull-secret podName:99ded651-5fce-49b7-8561-d54826fd2690 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:25.744260743 +0000 UTC m=+3.072833041 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/99ded651-5fce-49b7-8561-d54826fd2690-original-pull-secret") pod "global-pull-secret-syncer-2lqtw" (UID: "99ded651-5fce-49b7-8561-d54826fd2690") : object "kube-system"/"original-pull-secret" not registered
Jul 01 14:39:25.244593 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.244325    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-system-cni-dir\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.244593 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.244196    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-host-var-lib-cni-bin\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.244593 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.244337    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-multus-cni-dir\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.244593 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.244371    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-host-var-lib-cni-multus\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.244593 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.244384    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/99ded651-5fce-49b7-8561-d54826fd2690-dbus\") pod \"global-pull-secret-syncer-2lqtw\" (UID: \"99ded651-5fce-49b7-8561-d54826fd2690\") " pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:25.245214 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.244401    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-multus-cni-dir\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.245214 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.244415    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-multus-daemon-config\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.245214 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.244435    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-multus-conf-dir\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.245214 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.244437    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-host-run-multus-certs\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.245214 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.244474    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-host-var-lib-cni-multus\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.245214 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.244563    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/49ff50e4-fbc4-4623-a5c5-458c628adef3-serviceca\") pod \"node-ca-hn47h\" (UID: \"49ff50e4-fbc4-4623-a5c5-458c628adef3\") " pod="openshift-image-registry/node-ca-hn47h"
Jul 01 14:39:25.246540 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.246520    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0710008c-6eee-4fd1-aac9-81df1341171f-agent-certs\") pod \"konnectivity-agent-qrcb9\" (UID: \"0710008c-6eee-4fd1-aac9-81df1341171f\") " pod="kube-system/konnectivity-agent-qrcb9"
Jul 01 14:39:25.255920 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.255856    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ww49\" (UniqueName: \"kubernetes.io/projected/49ff50e4-fbc4-4623-a5c5-458c628adef3-kube-api-access-5ww49\") pod \"node-ca-hn47h\" (UID: \"49ff50e4-fbc4-4623-a5c5-458c628adef3\") " pod="openshift-image-registry/node-ca-hn47h"
Jul 01 14:39:25.256035 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.255918    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57mfr\" (UniqueName: \"kubernetes.io/projected/dc2cfa4b-185e-424a-af8b-c715dbe39c9d-kube-api-access-57mfr\") pod \"multus-jxqmr\" (UID: \"dc2cfa4b-185e-424a-af8b-c715dbe39c9d\") " pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.329224 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.328237    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mgl6l"
Jul 01 14:39:25.335806 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.335777    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jrwp"
Jul 01 14:39:25.344598 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.344569    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-649r6"
Jul 01 14:39:25.350685 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.350659    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rzw6z"
Jul 01 14:39:25.356807 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.356782    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nhhr7"
Jul 01 14:39:25.363976 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.363951    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:25.370998 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.370974    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qrcb9"
Jul 01 14:39:25.377086 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.377063    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jxqmr"
Jul 01 14:39:25.382966 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.382937    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hn47h"
Jul 01 14:39:25.383074 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.383026    2551 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 14:39:25.646701 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.646603    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/005968bd-f145-4c81-be95-b3170cafef84-metrics-certs\") pod \"network-metrics-daemon-tprn9\" (UID: \"005968bd-f145-4c81-be95-b3170cafef84\") " pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:39:25.646858 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:25.646757    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 14:39:25.646858 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:25.646827    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/005968bd-f145-4c81-be95-b3170cafef84-metrics-certs podName:005968bd-f145-4c81-be95-b3170cafef84 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:26.646811886 +0000 UTC m=+3.975384177 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/005968bd-f145-4c81-be95-b3170cafef84-metrics-certs") pod "network-metrics-daemon-tprn9" (UID: "005968bd-f145-4c81-be95-b3170cafef84") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 14:39:25.747267 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.747220    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/99ded651-5fce-49b7-8561-d54826fd2690-original-pull-secret\") pod \"global-pull-secret-syncer-2lqtw\" (UID: \"99ded651-5fce-49b7-8561-d54826fd2690\") " pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:25.747463 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:25.747278    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsszh\" (UniqueName: \"kubernetes.io/projected/8b8d9348-6896-4fd2-b3c1-8d841219bc43-kube-api-access-tsszh\") pod \"network-check-target-km6cc\" (UID: \"8b8d9348-6896-4fd2-b3c1-8d841219bc43\") " pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:39:25.747463 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:25.747394    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 14:39:25.747463 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:25.747404    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 14:39:25.747463 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:25.747427    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 14:39:25.747463 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:25.747438    2551 projected.go:196] Error preparing data for projected volume kube-api-access-tsszh for pod openshift-network-diagnostics/network-check-target-km6cc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 14:39:25.747463 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:25.747466    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99ded651-5fce-49b7-8561-d54826fd2690-original-pull-secret podName:99ded651-5fce-49b7-8561-d54826fd2690 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:26.747450179 +0000 UTC m=+4.076022475 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/99ded651-5fce-49b7-8561-d54826fd2690-original-pull-secret") pod "global-pull-secret-syncer-2lqtw" (UID: "99ded651-5fce-49b7-8561-d54826fd2690") : object "kube-system"/"original-pull-secret" not registered
Jul 01 14:39:25.747788 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:25.747484    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b8d9348-6896-4fd2-b3c1-8d841219bc43-kube-api-access-tsszh podName:8b8d9348-6896-4fd2-b3c1-8d841219bc43 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:26.74747276 +0000 UTC m=+4.076045044 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tsszh" (UniqueName: "kubernetes.io/projected/8b8d9348-6896-4fd2-b3c1-8d841219bc43-kube-api-access-tsszh") pod "network-check-target-km6cc" (UID: "8b8d9348-6896-4fd2-b3c1-8d841219bc43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 14:39:25.887188 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:25.887155    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc612921_fe17_41c3_93ba_2e1197f6f843.slice/crio-6af2b0164693efda1ab500b7a67cfc97f4b859772ec1216fd66247caf6f76da0 WatchSource:0}: Error finding container 6af2b0164693efda1ab500b7a67cfc97f4b859772ec1216fd66247caf6f76da0: Status 404 returned error can't find the container with id 6af2b0164693efda1ab500b7a67cfc97f4b859772ec1216fd66247caf6f76da0
Jul 01 14:39:25.889307 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:25.889286    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bc04385_5d77_4673_a019_88bb93a250dd.slice/crio-15228e105dcde6fa743fcf0d3d8758e4a3324fa42a120e325a2bae76dcd71184 WatchSource:0}: Error finding container 15228e105dcde6fa743fcf0d3d8758e4a3324fa42a120e325a2bae76dcd71184: Status 404 returned error can't find the container with id 15228e105dcde6fa743fcf0d3d8758e4a3324fa42a120e325a2bae76dcd71184
Jul 01 14:39:25.891130 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:25.891107    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49ff50e4_fbc4_4623_a5c5_458c628adef3.slice/crio-239de7cda4b4b7ef4849ffd8af161ec7a13da2cd1f2fc7231e168704ef888371 WatchSource:0}: Error finding container 239de7cda4b4b7ef4849ffd8af161ec7a13da2cd1f2fc7231e168704ef888371: Status 404 returned error can't find the container with id 239de7cda4b4b7ef4849ffd8af161ec7a13da2cd1f2fc7231e168704ef888371
Jul 01 14:39:25.892111 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:25.892087    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0710008c_6eee_4fd1_aac9_81df1341171f.slice/crio-0193976673ba3ad8db1a43c14b5608f4d7f6f8661f2659f40d276c8a874fc071 WatchSource:0}: Error finding container 0193976673ba3ad8db1a43c14b5608f4d7f6f8661f2659f40d276c8a874fc071: Status 404 returned error can't find the container with id 0193976673ba3ad8db1a43c14b5608f4d7f6f8661f2659f40d276c8a874fc071
Jul 01 14:39:25.893354 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:25.893332    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod768a96c0_935c_4629_b015_1b85b0ac3199.slice/crio-2e20043d0f160e76354d475123d788c2518f3fd602eb4eedd7d29894a2f91959 WatchSource:0}: Error finding container 2e20043d0f160e76354d475123d788c2518f3fd602eb4eedd7d29894a2f91959: Status 404 returned error can't find the container with id 2e20043d0f160e76354d475123d788c2518f3fd602eb4eedd7d29894a2f91959
Jul 01 14:39:25.895234 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:25.895165    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ea52ec7_825f_4d91_9215_17d64f0b497d.slice/crio-e7edfd33d5d7e4aa8eb3487481e61238eeb1c8a17cc4c96204022d16bb5d88e7 WatchSource:0}: Error finding container e7edfd33d5d7e4aa8eb3487481e61238eeb1c8a17cc4c96204022d16bb5d88e7: Status 404 returned error can't find the container with id e7edfd33d5d7e4aa8eb3487481e61238eeb1c8a17cc4c96204022d16bb5d88e7
Jul 01 14:39:25.896494 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:25.896465    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa1455f9_1f95_47e0_9d5f_f73c917257cf.slice/crio-6363e963784a67fce686b1e922fb19df679cb23df59ac8e4f355ab9175f0946d WatchSource:0}: Error finding container 6363e963784a67fce686b1e922fb19df679cb23df59ac8e4f355ab9175f0946d: Status 404 returned error can't find the container with id 6363e963784a67fce686b1e922fb19df679cb23df59ac8e4f355ab9175f0946d
Jul 01 14:39:25.898161 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:25.897675    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b10226f_d639_4a54_b33f_8b1cb9e35d17.slice/crio-25a22a167eff6de6b6beff716d2876725c45e9b850d2378681a91f7aa9b0288e WatchSource:0}: Error finding container 25a22a167eff6de6b6beff716d2876725c45e9b850d2378681a91f7aa9b0288e: Status 404 returned error can't find the container with id 25a22a167eff6de6b6beff716d2876725c45e9b850d2378681a91f7aa9b0288e
Jul 01 14:39:25.898233 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:25.898189    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc2cfa4b_185e_424a_af8b_c715dbe39c9d.slice/crio-b80268db506b6a66dcefff977dcc4418939613d3455549f3c889865ae535aab5 WatchSource:0}: Error finding container b80268db506b6a66dcefff977dcc4418939613d3455549f3c889865ae535aab5: Status 404 returned error can't find the container with id b80268db506b6a66dcefff977dcc4418939613d3455549f3c889865ae535aab5
Jul 01 14:39:26.078000 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:26.077766    2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 14:34:25 +0000 UTC" deadline="2028-01-11 15:25:56.585502497 +0000 UTC"
Jul 01 14:39:26.078000 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:26.077996    2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13416h46m30.507510644s"
Jul 01 14:39:26.106716 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:26.106660    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jrwp" event={"ID":"6b10226f-d639-4a54-b33f-8b1cb9e35d17","Type":"ContainerStarted","Data":"25a22a167eff6de6b6beff716d2876725c45e9b850d2378681a91f7aa9b0288e"}
Jul 01 14:39:26.107783 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:26.107753    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-649r6" event={"ID":"fa1455f9-1f95-47e0-9d5f-f73c917257cf","Type":"ContainerStarted","Data":"6363e963784a67fce686b1e922fb19df679cb23df59ac8e4f355ab9175f0946d"}
Jul 01 14:39:26.109292 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:26.109267    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mgl6l" event={"ID":"3ea52ec7-825f-4d91-9215-17d64f0b497d","Type":"ContainerStarted","Data":"e7edfd33d5d7e4aa8eb3487481e61238eeb1c8a17cc4c96204022d16bb5d88e7"}
Jul 01 14:39:26.111846 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:26.111811    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qnczw" event={"ID":"768a96c0-935c-4629-b015-1b85b0ac3199","Type":"ContainerStarted","Data":"2e20043d0f160e76354d475123d788c2518f3fd602eb4eedd7d29894a2f91959"}
Jul 01 14:39:26.112780 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:26.112750    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rzw6z" event={"ID":"4bc04385-5d77-4673-a019-88bb93a250dd","Type":"ContainerStarted","Data":"15228e105dcde6fa743fcf0d3d8758e4a3324fa42a120e325a2bae76dcd71184"}
Jul 01 14:39:26.114741 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:26.114718    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-55.ec2.internal" event={"ID":"814e0bac5e0890277234107c5ef5089c","Type":"ContainerStarted","Data":"10d258a056e282e922ee991bff7f83460dfddc201aa3f0e6407766734e3a8d03"}
Jul 01 14:39:26.115931 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:26.115909    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jxqmr" event={"ID":"dc2cfa4b-185e-424a-af8b-c715dbe39c9d","Type":"ContainerStarted","Data":"b80268db506b6a66dcefff977dcc4418939613d3455549f3c889865ae535aab5"}
Jul 01 14:39:26.117009 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:26.116991    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qrcb9" event={"ID":"0710008c-6eee-4fd1-aac9-81df1341171f","Type":"ContainerStarted","Data":"0193976673ba3ad8db1a43c14b5608f4d7f6f8661f2659f40d276c8a874fc071"}
Jul 01 14:39:26.117977 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:26.117950    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hn47h" event={"ID":"49ff50e4-fbc4-4623-a5c5-458c628adef3","Type":"ContainerStarted","Data":"239de7cda4b4b7ef4849ffd8af161ec7a13da2cd1f2fc7231e168704ef888371"}
Jul 01 14:39:26.119013 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:26.118983    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nhhr7" event={"ID":"bc612921-fe17-41c3-93ba-2e1197f6f843","Type":"ContainerStarted","Data":"6af2b0164693efda1ab500b7a67cfc97f4b859772ec1216fd66247caf6f76da0"}
Jul 01 14:39:26.140198 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:26.140138    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-55.ec2.internal" podStartSLOduration=2.140120135 podStartE2EDuration="2.140120135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 14:39:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:39:26.13992605 +0000 UTC m=+3.468498350" watchObservedRunningTime="2026-07-01 14:39:26.140120135 +0000 UTC m=+3.468692442"
Jul 01 14:39:26.653839 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:26.653715    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/005968bd-f145-4c81-be95-b3170cafef84-metrics-certs\") pod \"network-metrics-daemon-tprn9\" (UID: \"005968bd-f145-4c81-be95-b3170cafef84\") " pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:39:26.654031 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:26.653960    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 14:39:26.654031 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:26.654021    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/005968bd-f145-4c81-be95-b3170cafef84-metrics-certs podName:005968bd-f145-4c81-be95-b3170cafef84 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:28.654002871 +0000 UTC m=+5.982575158 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/005968bd-f145-4c81-be95-b3170cafef84-metrics-certs") pod "network-metrics-daemon-tprn9" (UID: "005968bd-f145-4c81-be95-b3170cafef84") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 14:39:26.754828 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:26.754741    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/99ded651-5fce-49b7-8561-d54826fd2690-original-pull-secret\") pod \"global-pull-secret-syncer-2lqtw\" (UID: \"99ded651-5fce-49b7-8561-d54826fd2690\") " pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:26.754828 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:26.754807    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsszh\" (UniqueName: \"kubernetes.io/projected/8b8d9348-6896-4fd2-b3c1-8d841219bc43-kube-api-access-tsszh\") pod \"network-check-target-km6cc\" (UID: \"8b8d9348-6896-4fd2-b3c1-8d841219bc43\") " pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:39:26.755047 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:26.754977    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 14:39:26.755047 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:26.755001    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 14:39:26.755047 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:26.755014    2551 projected.go:196] Error preparing data for projected volume kube-api-access-tsszh for pod openshift-network-diagnostics/network-check-target-km6cc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 14:39:26.755261 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:26.755074    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b8d9348-6896-4fd2-b3c1-8d841219bc43-kube-api-access-tsszh podName:8b8d9348-6896-4fd2-b3c1-8d841219bc43 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:28.755056441 +0000 UTC m=+6.083628728 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tsszh" (UniqueName: "kubernetes.io/projected/8b8d9348-6896-4fd2-b3c1-8d841219bc43-kube-api-access-tsszh") pod "network-check-target-km6cc" (UID: "8b8d9348-6896-4fd2-b3c1-8d841219bc43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 14:39:26.755766 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:26.755742    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 14:39:26.755873 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:26.755802    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99ded651-5fce-49b7-8561-d54826fd2690-original-pull-secret podName:99ded651-5fce-49b7-8561-d54826fd2690 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:28.755786823 +0000 UTC m=+6.084359112 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/99ded651-5fce-49b7-8561-d54826fd2690-original-pull-secret") pod "global-pull-secret-syncer-2lqtw" (UID: "99ded651-5fce-49b7-8561-d54826fd2690") : object "kube-system"/"original-pull-secret" not registered
Jul 01 14:39:27.078404 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:27.078303    2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 14:34:25 +0000 UTC" deadline="2028-02-08 07:17:22.281283818 +0000 UTC"
Jul 01 14:39:27.078404 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:27.078349    2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14080h37m55.202940001s"
Jul 01 14:39:27.099898 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:27.099826    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:39:27.099898 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:27.099857    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:27.100141 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:27.099964    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km6cc" podUID="8b8d9348-6896-4fd2-b3c1-8d841219bc43"
Jul 01 14:39:27.100404 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:27.099826    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:39:27.100404 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:27.100351    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2lqtw" podUID="99ded651-5fce-49b7-8561-d54826fd2690"
Jul 01 14:39:27.100657 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:27.100465    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tprn9" podUID="005968bd-f145-4c81-be95-b3170cafef84"
Jul 01 14:39:27.124603 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:27.124550    2551 generic.go:356] "Generic (PLEG): container finished" podID="f26db238ea81eafaa3ded9fde0688757" containerID="ccb9f043f685d8a39c85794209a90eb1be92863fc3529f9a586fcb0b4b91f277" exitCode=0
Jul 01 14:39:27.125768 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:27.125727    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal" event={"ID":"f26db238ea81eafaa3ded9fde0688757","Type":"ContainerDied","Data":"ccb9f043f685d8a39c85794209a90eb1be92863fc3529f9a586fcb0b4b91f277"}
Jul 01 14:39:28.130245 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:28.130184    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal" event={"ID":"f26db238ea81eafaa3ded9fde0688757","Type":"ContainerStarted","Data":"b0a77c61660fb4822e0ef61da57ce60573a62e8747c60688453d67ffcd733223"}
Jul 01 14:39:28.158080 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:28.158016    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal" podStartSLOduration=4.157996605 podStartE2EDuration="4.157996605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 14:39:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:39:28.157818912 +0000 UTC m=+5.486391219" watchObservedRunningTime="2026-07-01 14:39:28.157996605 +0000 UTC m=+5.486568910"
Jul 01 14:39:28.670677 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:28.670615    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/005968bd-f145-4c81-be95-b3170cafef84-metrics-certs\") pod \"network-metrics-daemon-tprn9\" (UID: \"005968bd-f145-4c81-be95-b3170cafef84\") " pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:39:28.670872 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:28.670796    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 14:39:28.670931 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:28.670879    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/005968bd-f145-4c81-be95-b3170cafef84-metrics-certs podName:005968bd-f145-4c81-be95-b3170cafef84 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:32.670857873 +0000 UTC m=+9.999430177 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/005968bd-f145-4c81-be95-b3170cafef84-metrics-certs") pod "network-metrics-daemon-tprn9" (UID: "005968bd-f145-4c81-be95-b3170cafef84") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 14:39:28.771177 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:28.771134    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/99ded651-5fce-49b7-8561-d54826fd2690-original-pull-secret\") pod \"global-pull-secret-syncer-2lqtw\" (UID: \"99ded651-5fce-49b7-8561-d54826fd2690\") " pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:28.771354 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:28.771202    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsszh\" (UniqueName: \"kubernetes.io/projected/8b8d9348-6896-4fd2-b3c1-8d841219bc43-kube-api-access-tsszh\") pod \"network-check-target-km6cc\" (UID: \"8b8d9348-6896-4fd2-b3c1-8d841219bc43\") " pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:39:28.771354 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:28.771330    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 14:39:28.771354 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:28.771342    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 14:39:28.771354 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:28.771357    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 14:39:28.771567 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:28.771369    2551 projected.go:196] Error preparing data for projected volume kube-api-access-tsszh for pod openshift-network-diagnostics/network-check-target-km6cc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 14:39:28.771567 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:28.771404    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99ded651-5fce-49b7-8561-d54826fd2690-original-pull-secret podName:99ded651-5fce-49b7-8561-d54826fd2690 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:32.771384379 +0000 UTC m=+10.099956676 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/99ded651-5fce-49b7-8561-d54826fd2690-original-pull-secret") pod "global-pull-secret-syncer-2lqtw" (UID: "99ded651-5fce-49b7-8561-d54826fd2690") : object "kube-system"/"original-pull-secret" not registered
Jul 01 14:39:28.771567 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:28.771423    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b8d9348-6896-4fd2-b3c1-8d841219bc43-kube-api-access-tsszh podName:8b8d9348-6896-4fd2-b3c1-8d841219bc43 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:32.771413944 +0000 UTC m=+10.099986234 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tsszh" (UniqueName: "kubernetes.io/projected/8b8d9348-6896-4fd2-b3c1-8d841219bc43-kube-api-access-tsszh") pod "network-check-target-km6cc" (UID: "8b8d9348-6896-4fd2-b3c1-8d841219bc43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 14:39:29.098667 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:29.098613    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:29.098870 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:29.098613    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:39:29.098870 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:29.098775    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:39:29.098870 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:29.098773    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2lqtw" podUID="99ded651-5fce-49b7-8561-d54826fd2690"
Jul 01 14:39:29.099033 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:29.098872    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km6cc" podUID="8b8d9348-6896-4fd2-b3c1-8d841219bc43"
Jul 01 14:39:29.099033 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:29.098974    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tprn9" podUID="005968bd-f145-4c81-be95-b3170cafef84"
Jul 01 14:39:31.098719 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:31.098634    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:39:31.098719 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:31.098659    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:39:31.099361 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:31.098780    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km6cc" podUID="8b8d9348-6896-4fd2-b3c1-8d841219bc43"
Jul 01 14:39:31.099361 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:31.098633    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:31.099361 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:31.099229    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2lqtw" podUID="99ded651-5fce-49b7-8561-d54826fd2690"
Jul 01 14:39:31.099361 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:31.099328    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tprn9" podUID="005968bd-f145-4c81-be95-b3170cafef84"
Jul 01 14:39:32.705192 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:32.705156    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/005968bd-f145-4c81-be95-b3170cafef84-metrics-certs\") pod \"network-metrics-daemon-tprn9\" (UID: \"005968bd-f145-4c81-be95-b3170cafef84\") " pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:39:32.711854 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:32.710846    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 14:39:32.711854 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:32.710936    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/005968bd-f145-4c81-be95-b3170cafef84-metrics-certs podName:005968bd-f145-4c81-be95-b3170cafef84 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:40.710915385 +0000 UTC m=+18.039487682 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/005968bd-f145-4c81-be95-b3170cafef84-metrics-certs") pod "network-metrics-daemon-tprn9" (UID: "005968bd-f145-4c81-be95-b3170cafef84") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 14:39:32.805931 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:32.805893    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/99ded651-5fce-49b7-8561-d54826fd2690-original-pull-secret\") pod \"global-pull-secret-syncer-2lqtw\" (UID: \"99ded651-5fce-49b7-8561-d54826fd2690\") " pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:32.806130 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:32.805959    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsszh\" (UniqueName: \"kubernetes.io/projected/8b8d9348-6896-4fd2-b3c1-8d841219bc43-kube-api-access-tsszh\") pod \"network-check-target-km6cc\" (UID: \"8b8d9348-6896-4fd2-b3c1-8d841219bc43\") " pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:39:32.806130 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:32.806120    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 14:39:32.806227 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:32.806141    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 14:39:32.806227 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:32.806155    2551 projected.go:196] Error preparing data for projected volume kube-api-access-tsszh for pod openshift-network-diagnostics/network-check-target-km6cc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 14:39:32.806227 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:32.806213    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b8d9348-6896-4fd2-b3c1-8d841219bc43-kube-api-access-tsszh podName:8b8d9348-6896-4fd2-b3c1-8d841219bc43 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:40.806194969 +0000 UTC m=+18.134767268 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tsszh" (UniqueName: "kubernetes.io/projected/8b8d9348-6896-4fd2-b3c1-8d841219bc43-kube-api-access-tsszh") pod "network-check-target-km6cc" (UID: "8b8d9348-6896-4fd2-b3c1-8d841219bc43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 14:39:32.806408 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:32.806291    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 14:39:32.806408 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:32.806324    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99ded651-5fce-49b7-8561-d54826fd2690-original-pull-secret podName:99ded651-5fce-49b7-8561-d54826fd2690 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:40.806313826 +0000 UTC m=+18.134886113 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/99ded651-5fce-49b7-8561-d54826fd2690-original-pull-secret") pod "global-pull-secret-syncer-2lqtw" (UID: "99ded651-5fce-49b7-8561-d54826fd2690") : object "kube-system"/"original-pull-secret" not registered
Jul 01 14:39:33.101038 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:33.099200    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:39:33.101038 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:33.099332    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tprn9" podUID="005968bd-f145-4c81-be95-b3170cafef84"
Jul 01 14:39:33.101038 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:33.099774    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:39:33.101038 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:33.099873    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km6cc" podUID="8b8d9348-6896-4fd2-b3c1-8d841219bc43"
Jul 01 14:39:33.101038 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:33.100007    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:33.101038 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:33.100094    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2lqtw" podUID="99ded651-5fce-49b7-8561-d54826fd2690"
Jul 01 14:39:35.100261 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:35.099523    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:35.100261 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:35.099673    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2lqtw" podUID="99ded651-5fce-49b7-8561-d54826fd2690"
Jul 01 14:39:35.100261 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:35.099732    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:39:35.100261 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:35.099797    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km6cc" podUID="8b8d9348-6896-4fd2-b3c1-8d841219bc43"
Jul 01 14:39:35.100261 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:35.099842    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:39:35.100261 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:35.099920    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tprn9" podUID="005968bd-f145-4c81-be95-b3170cafef84"
Jul 01 14:39:37.098271 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:37.098142    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:39:37.098752 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:37.098541    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tprn9" podUID="005968bd-f145-4c81-be95-b3170cafef84"
Jul 01 14:39:37.098752 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:37.098638    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:39:37.098851 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:37.098750    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km6cc" podUID="8b8d9348-6896-4fd2-b3c1-8d841219bc43"
Jul 01 14:39:37.100156 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:37.099899    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:37.101310 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:37.100340    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2lqtw" podUID="99ded651-5fce-49b7-8561-d54826fd2690"
Jul 01 14:39:39.098380 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:39.098343    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:39:39.098380 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:39.098378    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:39:39.098931 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:39.098344    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:39.098931 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:39.098478    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km6cc" podUID="8b8d9348-6896-4fd2-b3c1-8d841219bc43"
Jul 01 14:39:39.098931 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:39.098589    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2lqtw" podUID="99ded651-5fce-49b7-8561-d54826fd2690"
Jul 01 14:39:39.098931 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:39.098672    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tprn9" podUID="005968bd-f145-4c81-be95-b3170cafef84"
Jul 01 14:39:40.765131 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:40.765082    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/005968bd-f145-4c81-be95-b3170cafef84-metrics-certs\") pod \"network-metrics-daemon-tprn9\" (UID: \"005968bd-f145-4c81-be95-b3170cafef84\") " pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:39:40.765596 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:40.765232    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 14:39:40.765596 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:40.765317    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/005968bd-f145-4c81-be95-b3170cafef84-metrics-certs podName:005968bd-f145-4c81-be95-b3170cafef84 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:56.765295977 +0000 UTC m=+34.093868276 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/005968bd-f145-4c81-be95-b3170cafef84-metrics-certs") pod "network-metrics-daemon-tprn9" (UID: "005968bd-f145-4c81-be95-b3170cafef84") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 14:39:40.866329 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:40.866288    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/99ded651-5fce-49b7-8561-d54826fd2690-original-pull-secret\") pod \"global-pull-secret-syncer-2lqtw\" (UID: \"99ded651-5fce-49b7-8561-d54826fd2690\") " pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:40.866488 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:40.866352    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsszh\" (UniqueName: \"kubernetes.io/projected/8b8d9348-6896-4fd2-b3c1-8d841219bc43-kube-api-access-tsszh\") pod \"network-check-target-km6cc\" (UID: \"8b8d9348-6896-4fd2-b3c1-8d841219bc43\") " pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:39:40.866488 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:40.866465    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 14:39:40.866592 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:40.866503    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 14:39:40.866592 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:40.866521    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 14:39:40.866592 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:40.866535    2551 projected.go:196] Error preparing data for projected volume kube-api-access-tsszh for pod openshift-network-diagnostics/network-check-target-km6cc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 14:39:40.866592 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:40.866550    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99ded651-5fce-49b7-8561-d54826fd2690-original-pull-secret podName:99ded651-5fce-49b7-8561-d54826fd2690 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:56.866527473 +0000 UTC m=+34.195099778 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/99ded651-5fce-49b7-8561-d54826fd2690-original-pull-secret") pod "global-pull-secret-syncer-2lqtw" (UID: "99ded651-5fce-49b7-8561-d54826fd2690") : object "kube-system"/"original-pull-secret" not registered
Jul 01 14:39:40.866592 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:40.866585    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b8d9348-6896-4fd2-b3c1-8d841219bc43-kube-api-access-tsszh podName:8b8d9348-6896-4fd2-b3c1-8d841219bc43 nodeName:}" failed. No retries permitted until 2026-07-01 14:39:56.866570456 +0000 UTC m=+34.195142747 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tsszh" (UniqueName: "kubernetes.io/projected/8b8d9348-6896-4fd2-b3c1-8d841219bc43-kube-api-access-tsszh") pod "network-check-target-km6cc" (UID: "8b8d9348-6896-4fd2-b3c1-8d841219bc43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 14:39:41.098664 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:41.098611    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:39:41.098853 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:41.098688    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:41.098853 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:41.098611    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:39:41.098853 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:41.098787    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tprn9" podUID="005968bd-f145-4c81-be95-b3170cafef84"
Jul 01 14:39:41.098994 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:41.098869    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km6cc" podUID="8b8d9348-6896-4fd2-b3c1-8d841219bc43"
Jul 01 14:39:41.099341 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:41.098959    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2lqtw" podUID="99ded651-5fce-49b7-8561-d54826fd2690"
Jul 01 14:39:43.099265 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:43.099224    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:39:43.099769 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:43.099321    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km6cc" podUID="8b8d9348-6896-4fd2-b3c1-8d841219bc43"
Jul 01 14:39:43.099769 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:43.099371    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:43.099769 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:43.099435    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:39:43.099769 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:43.099540    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tprn9" podUID="005968bd-f145-4c81-be95-b3170cafef84"
Jul 01 14:39:43.099769 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:43.099634    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2lqtw" podUID="99ded651-5fce-49b7-8561-d54826fd2690"
Jul 01 14:39:45.098664 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:45.098258    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:39:45.099397 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:45.098262    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:39:45.099397 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:45.098770    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km6cc" podUID="8b8d9348-6896-4fd2-b3c1-8d841219bc43"
Jul 01 14:39:45.099397 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:45.098312    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:45.099397 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:45.098818    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tprn9" podUID="005968bd-f145-4c81-be95-b3170cafef84"
Jul 01 14:39:45.099397 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:45.098918    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2lqtw" podUID="99ded651-5fce-49b7-8561-d54826fd2690"
Jul 01 14:39:45.165265 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:45.165178    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rzw6z" event={"ID":"4bc04385-5d77-4673-a019-88bb93a250dd","Type":"ContainerStarted","Data":"278331885bf905438a3bb88c4a62c5b52246afa0055178f479d13f16b6a2a1a2"}
Jul 01 14:39:45.167332 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:45.167300    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jxqmr" event={"ID":"dc2cfa4b-185e-424a-af8b-c715dbe39c9d","Type":"ContainerStarted","Data":"7297b5fabdb5070bcd50963d27340cadf43a2980af1f42eb5aee24d726836b70"}
Jul 01 14:39:45.168993 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:45.168964    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qrcb9" event={"ID":"0710008c-6eee-4fd1-aac9-81df1341171f","Type":"ContainerStarted","Data":"14ff1f441e3f1dbd6c1693c6419d1d7e9766b403d516993a5d93d64081ac349d"}
Jul 01 14:39:45.170594 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:45.170559    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hn47h" event={"ID":"49ff50e4-fbc4-4623-a5c5-458c628adef3","Type":"ContainerStarted","Data":"2ba7f5067485d67b477de68cafb8939d90378a3e09df249bdffd1e82554ea7aa"}
Jul 01 14:39:45.172174 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:45.172149    2551 generic.go:356] "Generic (PLEG): container finished" podID="bc612921-fe17-41c3-93ba-2e1197f6f843" containerID="14b5150960a56125232cc2e8803b7648dc5bd4bd572b2bd48878aedc493a2d2f" exitCode=0
Jul 01 14:39:45.172319 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:45.172233    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nhhr7" event={"ID":"bc612921-fe17-41c3-93ba-2e1197f6f843","Type":"ContainerDied","Data":"14b5150960a56125232cc2e8803b7648dc5bd4bd572b2bd48878aedc493a2d2f"}
Jul 01 14:39:45.174348 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:45.174315    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jrwp" event={"ID":"6b10226f-d639-4a54-b33f-8b1cb9e35d17","Type":"ContainerStarted","Data":"e40a77690741b7ef81c229ccc36c01991dd7e52266014931e1da68a04545d4e1"}
Jul 01 14:39:45.176194 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:45.176144    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-649r6" event={"ID":"fa1455f9-1f95-47e0-9d5f-f73c917257cf","Type":"ContainerStarted","Data":"3baec01228904c49324435ce74556dce2912ba8b47ffb915e0b0e8cedb7c75e5"}
Jul 01 14:39:45.179359 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:45.179339    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/ovn-acl-logging/0.log"
Jul 01 14:39:45.179839 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:45.179813    2551 generic.go:356] "Generic (PLEG): container finished" podID="768a96c0-935c-4629-b015-1b85b0ac3199" containerID="1dd86d695882676fae6ec02cb68a9a18b7aee790a6af0d8f299bf82bb3a3311d" exitCode=1
Jul 01 14:39:45.179930 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:45.179853    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qnczw" event={"ID":"768a96c0-935c-4629-b015-1b85b0ac3199","Type":"ContainerStarted","Data":"4b1b1c40237c417cd8160345972410993c89e9b2c9a5ff630cc8166808cb3918"}
Jul 01 14:39:45.179930 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:45.179921    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qnczw" event={"ID":"768a96c0-935c-4629-b015-1b85b0ac3199","Type":"ContainerStarted","Data":"fdf757953dd22d331bff939f2a42a010d8d7e10f86f50a5352cd34a43886eb09"}
Jul 01 14:39:45.180040 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:45.179971    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qnczw" event={"ID":"768a96c0-935c-4629-b015-1b85b0ac3199","Type":"ContainerStarted","Data":"ebc8573e5e1ae5d35a54dd54eb09eedde7b97c308b1e128ab1f0db075c61754a"}
Jul 01 14:39:45.180040 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:45.180018    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qnczw" event={"ID":"768a96c0-935c-4629-b015-1b85b0ac3199","Type":"ContainerStarted","Data":"6df73ca1fae5f8c7757099e0d29fbc1eaeca0f4c9af14b681c145fdf204b42ad"}
Jul 01 14:39:45.180134 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:45.180066    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qnczw" event={"ID":"768a96c0-935c-4629-b015-1b85b0ac3199","Type":"ContainerDied","Data":"1dd86d695882676fae6ec02cb68a9a18b7aee790a6af0d8f299bf82bb3a3311d"}
Jul 01 14:39:45.180134 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:45.180115    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qnczw" event={"ID":"768a96c0-935c-4629-b015-1b85b0ac3199","Type":"ContainerStarted","Data":"a3ef1f06ced9e4a36b6ac55a7a32a6ae03987e21e95acf9d19376a697afeda32"}
Jul 01 14:39:45.184720 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:45.184671    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rzw6z" podStartSLOduration=3.694692688 podStartE2EDuration="22.184655478s" totalImagesPullingTime="18.48996279s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 14:39:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:39:45.184366504 +0000 UTC m=+22.512938810" watchObservedRunningTime="2026-07-01 14:39:45.184655478 +0000 UTC m=+22.513227785"
Jul 01 14:39:45.205417 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:45.205368    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qrcb9" podStartSLOduration=3.714816113 podStartE2EDuration="22.20534239s" totalImagesPullingTime="18.490526277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:39:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:39:45.204966866 +0000 UTC m=+22.533539173" watchObservedRunningTime="2026-07-01 14:39:45.20534239 +0000 UTC m=+22.533914696"
Jul 01 14:39:45.228337 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:45.228281    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-hn47h" podStartSLOduration=7.4775525720000005 podStartE2EDuration="22.228264984s" totalImagesPullingTime="14.750712412s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 14:39:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:39:45.227739235 +0000 UTC m=+22.556311543" watchObservedRunningTime="2026-07-01 14:39:45.228264984 +0000 UTC m=+22.556837289"
Jul 01 14:39:45.250243 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:45.250181    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-649r6" podStartSLOduration=3.765304558 podStartE2EDuration="22.250159342s" totalImagesPullingTime="18.484854784s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 14:39:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:39:45.250104285 +0000 UTC m=+22.578676591" watchObservedRunningTime="2026-07-01 14:39:45.250159342 +0000 UTC m=+22.578731644"
Jul 01 14:39:45.279823 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:45.279751    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jxqmr" podStartSLOduration=3.729341691 podStartE2EDuration="22.279728346s" totalImagesPullingTime="18.550386655s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 14:39:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:39:45.279211112 +0000 UTC m=+22.607783419" watchObservedRunningTime="2026-07-01 14:39:45.279728346 +0000 UTC m=+22.608300653"
Jul 01 14:39:45.529216 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:45.529191    2551 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 01 14:39:46.093832 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:46.093651    2551 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-01T14:39:45.529213018Z","UUID":"c1cface6-dd55-403f-a6b2-980692138a18","Handler":null,"Name":"","Endpoint":""}
Jul 01 14:39:46.096559 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:46.096411    2551 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 01 14:39:46.096559 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:46.096484    2551 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 01 14:39:46.185050 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:46.185015    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jrwp" event={"ID":"6b10226f-d639-4a54-b33f-8b1cb9e35d17","Type":"ContainerStarted","Data":"bfccc9e184a9a1c9bb30ed09976289bfeb0ecba9063f4bece28443a3d1693849"}
Jul 01 14:39:46.186571 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:46.186534    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mgl6l" event={"ID":"3ea52ec7-825f-4d91-9215-17d64f0b497d","Type":"ContainerStarted","Data":"defd412bcaf77ccaae8a7d7617109a488f93fc7a87c22854fa96472e10bfd701"}
Jul 01 14:39:47.098972 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:47.098779    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:39:47.098972 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:47.098815    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:39:47.098972 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:47.098950    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tprn9" podUID="005968bd-f145-4c81-be95-b3170cafef84"
Jul 01 14:39:47.099265 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:47.098997    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:47.099265 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:47.099115    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km6cc" podUID="8b8d9348-6896-4fd2-b3c1-8d841219bc43"
Jul 01 14:39:47.099265 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:47.099199    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2lqtw" podUID="99ded651-5fce-49b7-8561-d54826fd2690"
Jul 01 14:39:47.193170 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:47.193146    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/ovn-acl-logging/0.log"
Jul 01 14:39:47.193607 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:47.193574    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qnczw" event={"ID":"768a96c0-935c-4629-b015-1b85b0ac3199","Type":"ContainerStarted","Data":"f5ef43f4de45ec7579f1dfa8548848c037915cb3540d95dbc1654eadf835cee2"}
Jul 01 14:39:49.098270 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:49.098030    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:49.098694 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:49.098079    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:39:49.098694 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:49.098375    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2lqtw" podUID="99ded651-5fce-49b7-8561-d54826fd2690"
Jul 01 14:39:49.098694 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:49.098100    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:39:49.098694 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:49.098416    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tprn9" podUID="005968bd-f145-4c81-be95-b3170cafef84"
Jul 01 14:39:49.098694 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:49.098507    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km6cc" podUID="8b8d9348-6896-4fd2-b3c1-8d841219bc43"
Jul 01 14:39:50.167067 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:50.166849    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qrcb9"
Jul 01 14:39:50.167866 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:50.167555    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qrcb9"
Jul 01 14:39:50.193506 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:50.193456    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mgl6l" podStartSLOduration=8.703373299999999 podStartE2EDuration="27.193439942s" totalImagesPullingTime="18.490066642s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 14:39:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:39:46.205288978 +0000 UTC m=+23.533861285" watchObservedRunningTime="2026-07-01 14:39:50.193439942 +0000 UTC m=+27.522012358"
Jul 01 14:39:50.201285 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:50.201253    2551 generic.go:356] "Generic (PLEG): container finished" podID="bc612921-fe17-41c3-93ba-2e1197f6f843" containerID="9d2373c616fee8a346d940eabf8ffc363c453ae2ebfbb0cba5379c288a6f9c59" exitCode=0
Jul 01 14:39:50.201414 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:50.201337    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nhhr7" event={"ID":"bc612921-fe17-41c3-93ba-2e1197f6f843","Type":"ContainerDied","Data":"9d2373c616fee8a346d940eabf8ffc363c453ae2ebfbb0cba5379c288a6f9c59"}
Jul 01 14:39:50.203510 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:50.203485    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jrwp" event={"ID":"6b10226f-d639-4a54-b33f-8b1cb9e35d17","Type":"ContainerStarted","Data":"9c96e7a6f033f86b75b52cafeb2f7754c76e5ee9eedef90d04bb2744c061b02a"}
Jul 01 14:39:50.206861 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:50.206842    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/ovn-acl-logging/0.log"
Jul 01 14:39:50.207185 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:50.207164    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qnczw" event={"ID":"768a96c0-935c-4629-b015-1b85b0ac3199","Type":"ContainerStarted","Data":"60b1e9730f894cc17ad12b70fa9a2e21ae550e5ad9f622963cd668362f7407df"}
Jul 01 14:39:50.207408 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:50.207389    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qrcb9"
Jul 01 14:39:50.207484 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:50.207448    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:50.207539 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:50.207488    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:50.207657 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:50.207613    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:50.207884 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:50.207790    2551 scope.go:117] "RemoveContainer" containerID="1dd86d695882676fae6ec02cb68a9a18b7aee790a6af0d8f299bf82bb3a3311d"
Jul 01 14:39:50.207961 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:50.207937    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qrcb9"
Jul 01 14:39:50.223034 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:50.223014    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:50.223122 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:50.223114    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:39:50.252863 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:50.252817    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4jrwp" podStartSLOduration=3.513375095 podStartE2EDuration="27.252801483s" totalImagesPullingTime="23.739426388s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 14:39:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:39:50.252317968 +0000 UTC m=+27.580890276" watchObservedRunningTime="2026-07-01 14:39:50.252801483 +0000 UTC m=+27.581373789"
Jul 01 14:39:51.097989 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:51.097927    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:39:51.097989 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:51.097954    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:51.098222 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:51.097928    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:39:51.098222 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:51.098081    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tprn9" podUID="005968bd-f145-4c81-be95-b3170cafef84"
Jul 01 14:39:51.098320 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:51.098216    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km6cc" podUID="8b8d9348-6896-4fd2-b3c1-8d841219bc43"
Jul 01 14:39:51.098320 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:51.098305    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2lqtw" podUID="99ded651-5fce-49b7-8561-d54826fd2690"
Jul 01 14:39:51.214984 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:51.214907    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/ovn-acl-logging/0.log"
Jul 01 14:39:51.215436 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:51.215348    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qnczw" event={"ID":"768a96c0-935c-4629-b015-1b85b0ac3199","Type":"ContainerStarted","Data":"4fc83f089cf22d0e12e0be8047bba5d6b116f453c78bb803a38f260aabd6af7a"}
Jul 01 14:39:51.457910 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:51.457852    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qnczw" podStartSLOduration=9.903079174 podStartE2EDuration="28.457833892s" totalImagesPullingTime="18.554754718s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 14:39:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:39:51.396281878 +0000 UTC m=+28.724854184" watchObservedRunningTime="2026-07-01 14:39:51.457833892 +0000 UTC m=+28.786406235"
Jul 01 14:39:51.458273 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:51.458235    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tprn9"]
Jul 01 14:39:51.458361 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:51.458345    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:39:51.458464 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:51.458440    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tprn9" podUID="005968bd-f145-4c81-be95-b3170cafef84"
Jul 01 14:39:51.460532 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:51.460502    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-km6cc"]
Jul 01 14:39:51.460645 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:51.460612    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:39:51.460810 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:51.460753    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km6cc" podUID="8b8d9348-6896-4fd2-b3c1-8d841219bc43"
Jul 01 14:39:51.461160 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:51.461137    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2lqtw"]
Jul 01 14:39:51.461246 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:51.461210    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:51.461319 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:51.461294    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2lqtw" podUID="99ded651-5fce-49b7-8561-d54826fd2690"
Jul 01 14:39:52.219284 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:52.219250    2551 generic.go:356] "Generic (PLEG): container finished" podID="bc612921-fe17-41c3-93ba-2e1197f6f843" containerID="a17303e1b055ed7e6d71745cd8cbdd850d02d4e11170a6ad1c224fd4b23b7497" exitCode=0
Jul 01 14:39:52.219757 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:52.219335    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nhhr7" event={"ID":"bc612921-fe17-41c3-93ba-2e1197f6f843","Type":"ContainerDied","Data":"a17303e1b055ed7e6d71745cd8cbdd850d02d4e11170a6ad1c224fd4b23b7497"}
Jul 01 14:39:53.098569 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:53.098533    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:53.098768 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:53.098650    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:39:53.098768 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:53.098672    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:39:53.098768 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:53.098644    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2lqtw" podUID="99ded651-5fce-49b7-8561-d54826fd2690"
Jul 01 14:39:53.098875 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:53.098790    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tprn9" podUID="005968bd-f145-4c81-be95-b3170cafef84"
Jul 01 14:39:53.098916 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:53.098882    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km6cc" podUID="8b8d9348-6896-4fd2-b3c1-8d841219bc43"
Jul 01 14:39:54.226873 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:54.226585    2551 generic.go:356] "Generic (PLEG): container finished" podID="bc612921-fe17-41c3-93ba-2e1197f6f843" containerID="0b82b21c800cacee182be9a11c050db3a117306446edbfb30cb098ff1f9cf0e3" exitCode=0
Jul 01 14:39:54.227398 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:54.226675    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nhhr7" event={"ID":"bc612921-fe17-41c3-93ba-2e1197f6f843","Type":"ContainerDied","Data":"0b82b21c800cacee182be9a11c050db3a117306446edbfb30cb098ff1f9cf0e3"}
Jul 01 14:39:55.098728 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:55.098693    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:39:55.098728 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:55.098714    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:55.098979 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:55.098702    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:39:55.098979 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:55.098836    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tprn9" podUID="005968bd-f145-4c81-be95-b3170cafef84"
Jul 01 14:39:55.098979 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:55.098894    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km6cc" podUID="8b8d9348-6896-4fd2-b3c1-8d841219bc43"
Jul 01 14:39:55.099135 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:55.098975    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2lqtw" podUID="99ded651-5fce-49b7-8561-d54826fd2690"
Jul 01 14:39:56.778659 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:56.778607    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/005968bd-f145-4c81-be95-b3170cafef84-metrics-certs\") pod \"network-metrics-daemon-tprn9\" (UID: \"005968bd-f145-4c81-be95-b3170cafef84\") " pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:39:56.779190 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:56.778769    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 14:39:56.779190 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:56.778840    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/005968bd-f145-4c81-be95-b3170cafef84-metrics-certs podName:005968bd-f145-4c81-be95-b3170cafef84 nodeName:}" failed. No retries permitted until 2026-07-01 14:40:28.778824043 +0000 UTC m=+66.107396327 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/005968bd-f145-4c81-be95-b3170cafef84-metrics-certs") pod "network-metrics-daemon-tprn9" (UID: "005968bd-f145-4c81-be95-b3170cafef84") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 14:39:56.879259 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:56.879225    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/99ded651-5fce-49b7-8561-d54826fd2690-original-pull-secret\") pod \"global-pull-secret-syncer-2lqtw\" (UID: \"99ded651-5fce-49b7-8561-d54826fd2690\") " pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:56.879259 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:56.879272    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsszh\" (UniqueName: \"kubernetes.io/projected/8b8d9348-6896-4fd2-b3c1-8d841219bc43-kube-api-access-tsszh\") pod \"network-check-target-km6cc\" (UID: \"8b8d9348-6896-4fd2-b3c1-8d841219bc43\") " pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:39:56.879480 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:56.879389    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 14:39:56.879480 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:56.879409    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 14:39:56.879480 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:56.879425    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 14:39:56.879480 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:56.879439    2551 projected.go:196] Error preparing data for projected volume kube-api-access-tsszh for pod openshift-network-diagnostics/network-check-target-km6cc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 14:39:56.879480 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:56.879461    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99ded651-5fce-49b7-8561-d54826fd2690-original-pull-secret podName:99ded651-5fce-49b7-8561-d54826fd2690 nodeName:}" failed. No retries permitted until 2026-07-01 14:40:28.879442702 +0000 UTC m=+66.208015004 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/99ded651-5fce-49b7-8561-d54826fd2690-original-pull-secret") pod "global-pull-secret-syncer-2lqtw" (UID: "99ded651-5fce-49b7-8561-d54826fd2690") : object "kube-system"/"original-pull-secret" not registered
Jul 01 14:39:56.879480 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:56.879477    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b8d9348-6896-4fd2-b3c1-8d841219bc43-kube-api-access-tsszh podName:8b8d9348-6896-4fd2-b3c1-8d841219bc43 nodeName:}" failed. No retries permitted until 2026-07-01 14:40:28.879467521 +0000 UTC m=+66.208039809 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tsszh" (UniqueName: "kubernetes.io/projected/8b8d9348-6896-4fd2-b3c1-8d841219bc43-kube-api-access-tsszh") pod "network-check-target-km6cc" (UID: "8b8d9348-6896-4fd2-b3c1-8d841219bc43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 14:39:57.098148 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.098059    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:39:57.098148 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.098094    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:57.098148 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.098059    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:39:57.098396 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:57.098203    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2lqtw" podUID="99ded651-5fce-49b7-8561-d54826fd2690"
Jul 01 14:39:57.098396 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:57.098298    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tprn9" podUID="005968bd-f145-4c81-be95-b3170cafef84"
Jul 01 14:39:57.098463 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:57.098396    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km6cc" podUID="8b8d9348-6896-4fd2-b3c1-8d841219bc43"
Jul 01 14:39:57.414367 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.414273    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeReady"
Jul 01 14:39:57.414550 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.414423    2551 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 01 14:39:57.457560 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.457522    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-67f9d68f9-xgrmg"]
Jul 01 14:39:57.490290 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.490257    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67f9d68f9-xgrmg"]
Jul 01 14:39:57.490290 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.490300    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9grqk"]
Jul 01 14:39:57.490525 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.490396    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:39:57.494041 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.493370    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 01 14:39:57.494041 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.493728    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 01 14:39:57.494041 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.493778    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-45ph4\""
Jul 01 14:39:57.496511 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.496473    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 01 14:39:57.510861 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.510820    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jw949"]
Jul 01 14:39:57.511473 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.511162    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9grqk"
Jul 01 14:39:57.512466 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.512443    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 01 14:39:57.515725 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.515692    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 01 14:39:57.515883 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.515740    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 01 14:39:57.516605 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.516477    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xs5sl\""
Jul 01 14:39:57.532000 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.531958    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jw949"]
Jul 01 14:39:57.532172 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.532013    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9grqk"]
Jul 01 14:39:57.532172 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.532104    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jw949"
Jul 01 14:39:57.535169 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.535145    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-58rmg\""
Jul 01 14:39:57.535862 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.535840    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 01 14:39:57.535985 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.535848    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 01 14:39:57.536153 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.536138    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 01 14:39:57.583150 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.583101    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1e94a284-d202-4352-82fa-dd51de2f2c4e-config-volume\") pod \"dns-default-9grqk\" (UID: \"1e94a284-d202-4352-82fa-dd51de2f2c4e\") " pod="openshift-dns/dns-default-9grqk"
Jul 01 14:39:57.583329 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.583203    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e94a284-d202-4352-82fa-dd51de2f2c4e-metrics-tls\") pod \"dns-default-9grqk\" (UID: \"1e94a284-d202-4352-82fa-dd51de2f2c4e\") " pod="openshift-dns/dns-default-9grqk"
Jul 01 14:39:57.583329 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.583252    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-registry-tls\") pod \"image-registry-67f9d68f9-xgrmg\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") " pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:39:57.583329 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.583278    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5z5rf\" (UniqueName: \"kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-kube-api-access-5z5rf\") pod \"image-registry-67f9d68f9-xgrmg\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") " pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:39:57.583329 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.583308    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f4470e2b-27f5-49b2-b979-e72a8e31eddc-ca-trust-extracted\") pod \"image-registry-67f9d68f9-xgrmg\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") " pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:39:57.583521 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.583340    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2259h\" (UniqueName: \"kubernetes.io/projected/1e94a284-d202-4352-82fa-dd51de2f2c4e-kube-api-access-2259h\") pod \"dns-default-9grqk\" (UID: \"1e94a284-d202-4352-82fa-dd51de2f2c4e\") " pod="openshift-dns/dns-default-9grqk"
Jul 01 14:39:57.583521 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.583365    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f4470e2b-27f5-49b2-b979-e72a8e31eddc-registry-certificates\") pod \"image-registry-67f9d68f9-xgrmg\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") " pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:39:57.583521 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.583391    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-bound-sa-token\") pod \"image-registry-67f9d68f9-xgrmg\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") " pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:39:57.583521 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.583472    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e94a284-d202-4352-82fa-dd51de2f2c4e-tmp-dir\") pod \"dns-default-9grqk\" (UID: \"1e94a284-d202-4352-82fa-dd51de2f2c4e\") " pod="openshift-dns/dns-default-9grqk"
Jul 01 14:39:57.583744 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.583516    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f4470e2b-27f5-49b2-b979-e72a8e31eddc-installation-pull-secrets\") pod \"image-registry-67f9d68f9-xgrmg\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") " pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:39:57.583744 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.583552    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f4470e2b-27f5-49b2-b979-e72a8e31eddc-image-registry-private-configuration\") pod \"image-registry-67f9d68f9-xgrmg\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") " pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:39:57.583744 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.583602    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f4470e2b-27f5-49b2-b979-e72a8e31eddc-trusted-ca\") pod \"image-registry-67f9d68f9-xgrmg\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") " pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:39:57.685070 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.684957    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f4470e2b-27f5-49b2-b979-e72a8e31eddc-trusted-ca\") pod \"image-registry-67f9d68f9-xgrmg\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") " pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:39:57.685070 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.685022    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1e94a284-d202-4352-82fa-dd51de2f2c4e-config-volume\") pod \"dns-default-9grqk\" (UID: \"1e94a284-d202-4352-82fa-dd51de2f2c4e\") " pod="openshift-dns/dns-default-9grqk"
Jul 01 14:39:57.685070 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.685051    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f93bdf31-79e7-4d7f-9956-5a5a7e9e5075-cert\") pod \"ingress-canary-jw949\" (UID: \"f93bdf31-79e7-4d7f-9956-5a5a7e9e5075\") " pod="openshift-ingress-canary/ingress-canary-jw949"
Jul 01 14:39:57.685355 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.685102    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e94a284-d202-4352-82fa-dd51de2f2c4e-metrics-tls\") pod \"dns-default-9grqk\" (UID: \"1e94a284-d202-4352-82fa-dd51de2f2c4e\") " pod="openshift-dns/dns-default-9grqk"
Jul 01 14:39:57.685355 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.685141    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-registry-tls\") pod \"image-registry-67f9d68f9-xgrmg\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") " pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:39:57.685355 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.685164    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5z5rf\" (UniqueName: \"kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-kube-api-access-5z5rf\") pod \"image-registry-67f9d68f9-xgrmg\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") " pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:39:57.685355 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.685201    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f4470e2b-27f5-49b2-b979-e72a8e31eddc-ca-trust-extracted\") pod \"image-registry-67f9d68f9-xgrmg\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") " pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:39:57.685355 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.685225    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9n66\" (UniqueName: \"kubernetes.io/projected/f93bdf31-79e7-4d7f-9956-5a5a7e9e5075-kube-api-access-c9n66\") pod \"ingress-canary-jw949\" (UID: \"f93bdf31-79e7-4d7f-9956-5a5a7e9e5075\") " pod="openshift-ingress-canary/ingress-canary-jw949"
Jul 01 14:39:57.685355 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.685258    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2259h\" (UniqueName: \"kubernetes.io/projected/1e94a284-d202-4352-82fa-dd51de2f2c4e-kube-api-access-2259h\") pod \"dns-default-9grqk\" (UID: \"1e94a284-d202-4352-82fa-dd51de2f2c4e\") " pod="openshift-dns/dns-default-9grqk"
Jul 01 14:39:57.685355 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:57.685261    2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 14:39:57.685355 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.685286    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f4470e2b-27f5-49b2-b979-e72a8e31eddc-registry-certificates\") pod \"image-registry-67f9d68f9-xgrmg\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") " pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:39:57.685355 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.685308    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-bound-sa-token\") pod \"image-registry-67f9d68f9-xgrmg\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") " pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:39:57.685355 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:57.685347    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e94a284-d202-4352-82fa-dd51de2f2c4e-metrics-tls podName:1e94a284-d202-4352-82fa-dd51de2f2c4e nodeName:}" failed. No retries permitted until 2026-07-01 14:39:58.185325013 +0000 UTC m=+35.513897299 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1e94a284-d202-4352-82fa-dd51de2f2c4e-metrics-tls") pod "dns-default-9grqk" (UID: "1e94a284-d202-4352-82fa-dd51de2f2c4e") : secret "dns-default-metrics-tls" not found
Jul 01 14:39:57.685836 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.685395    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e94a284-d202-4352-82fa-dd51de2f2c4e-tmp-dir\") pod \"dns-default-9grqk\" (UID: \"1e94a284-d202-4352-82fa-dd51de2f2c4e\") " pod="openshift-dns/dns-default-9grqk"
Jul 01 14:39:57.685836 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.685427    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f4470e2b-27f5-49b2-b979-e72a8e31eddc-installation-pull-secrets\") pod \"image-registry-67f9d68f9-xgrmg\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") " pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:39:57.685836 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.685466    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f4470e2b-27f5-49b2-b979-e72a8e31eddc-image-registry-private-configuration\") pod \"image-registry-67f9d68f9-xgrmg\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") " pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:39:57.685836 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.685586    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f4470e2b-27f5-49b2-b979-e72a8e31eddc-ca-trust-extracted\") pod \"image-registry-67f9d68f9-xgrmg\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") " pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:39:57.685836 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:57.685261    2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 14:39:57.685836 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:57.685663    2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67f9d68f9-xgrmg: secret "image-registry-tls" not found
Jul 01 14:39:57.685836 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.685809    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1e94a284-d202-4352-82fa-dd51de2f2c4e-config-volume\") pod \"dns-default-9grqk\" (UID: \"1e94a284-d202-4352-82fa-dd51de2f2c4e\") " pod="openshift-dns/dns-default-9grqk"
Jul 01 14:39:57.685836 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:57.685840    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-registry-tls podName:f4470e2b-27f5-49b2-b979-e72a8e31eddc nodeName:}" failed. No retries permitted until 2026-07-01 14:39:58.185819004 +0000 UTC m=+35.514391296 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-registry-tls") pod "image-registry-67f9d68f9-xgrmg" (UID: "f4470e2b-27f5-49b2-b979-e72a8e31eddc") : secret "image-registry-tls" not found
Jul 01 14:39:57.686182 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.686009    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e94a284-d202-4352-82fa-dd51de2f2c4e-tmp-dir\") pod \"dns-default-9grqk\" (UID: \"1e94a284-d202-4352-82fa-dd51de2f2c4e\") " pod="openshift-dns/dns-default-9grqk"
Jul 01 14:39:57.686182 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.686013    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f4470e2b-27f5-49b2-b979-e72a8e31eddc-registry-certificates\") pod \"image-registry-67f9d68f9-xgrmg\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") " pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:39:57.686273 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.686187    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f4470e2b-27f5-49b2-b979-e72a8e31eddc-trusted-ca\") pod \"image-registry-67f9d68f9-xgrmg\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") " pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:39:57.691204 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.691180    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f4470e2b-27f5-49b2-b979-e72a8e31eddc-image-registry-private-configuration\") pod \"image-registry-67f9d68f9-xgrmg\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") " pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:39:57.691352 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.691266    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f4470e2b-27f5-49b2-b979-e72a8e31eddc-installation-pull-secrets\") pod \"image-registry-67f9d68f9-xgrmg\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") " pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:39:57.695259 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.695235    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2259h\" (UniqueName: \"kubernetes.io/projected/1e94a284-d202-4352-82fa-dd51de2f2c4e-kube-api-access-2259h\") pod \"dns-default-9grqk\" (UID: \"1e94a284-d202-4352-82fa-dd51de2f2c4e\") " pod="openshift-dns/dns-default-9grqk"
Jul 01 14:39:57.695818 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.695794    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-bound-sa-token\") pod \"image-registry-67f9d68f9-xgrmg\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") " pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:39:57.695941 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.695793    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5z5rf\" (UniqueName: \"kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-kube-api-access-5z5rf\") pod \"image-registry-67f9d68f9-xgrmg\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") " pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:39:57.786442 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.786406    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f93bdf31-79e7-4d7f-9956-5a5a7e9e5075-cert\") pod \"ingress-canary-jw949\" (UID: \"f93bdf31-79e7-4d7f-9956-5a5a7e9e5075\") " pod="openshift-ingress-canary/ingress-canary-jw949"
Jul 01 14:39:57.787109 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.786490    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9n66\" (UniqueName: \"kubernetes.io/projected/f93bdf31-79e7-4d7f-9956-5a5a7e9e5075-kube-api-access-c9n66\") pod \"ingress-canary-jw949\" (UID: \"f93bdf31-79e7-4d7f-9956-5a5a7e9e5075\") " pod="openshift-ingress-canary/ingress-canary-jw949"
Jul 01 14:39:57.789488 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.789452    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f93bdf31-79e7-4d7f-9956-5a5a7e9e5075-cert\") pod \"ingress-canary-jw949\" (UID: \"f93bdf31-79e7-4d7f-9956-5a5a7e9e5075\") " pod="openshift-ingress-canary/ingress-canary-jw949"
Jul 01 14:39:57.796485 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.796458    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9n66\" (UniqueName: \"kubernetes.io/projected/f93bdf31-79e7-4d7f-9956-5a5a7e9e5075-kube-api-access-c9n66\") pod \"ingress-canary-jw949\" (UID: \"f93bdf31-79e7-4d7f-9956-5a5a7e9e5075\") " pod="openshift-ingress-canary/ingress-canary-jw949"
Jul 01 14:39:57.843571 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:57.843528    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jw949"
Jul 01 14:39:58.190536 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:58.190498    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e94a284-d202-4352-82fa-dd51de2f2c4e-metrics-tls\") pod \"dns-default-9grqk\" (UID: \"1e94a284-d202-4352-82fa-dd51de2f2c4e\") " pod="openshift-dns/dns-default-9grqk"
Jul 01 14:39:58.190737 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:58.190546    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-registry-tls\") pod \"image-registry-67f9d68f9-xgrmg\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") " pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:39:58.190737 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:58.190690    2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 14:39:58.190737 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:58.190714    2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67f9d68f9-xgrmg: secret "image-registry-tls" not found
Jul 01 14:39:58.190881 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:58.190765    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-registry-tls podName:f4470e2b-27f5-49b2-b979-e72a8e31eddc nodeName:}" failed. No retries permitted until 2026-07-01 14:39:59.190751112 +0000 UTC m=+36.519323396 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-registry-tls") pod "image-registry-67f9d68f9-xgrmg" (UID: "f4470e2b-27f5-49b2-b979-e72a8e31eddc") : secret "image-registry-tls" not found
Jul 01 14:39:58.190881 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:58.190690    2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 14:39:58.190881 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:58.190837    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e94a284-d202-4352-82fa-dd51de2f2c4e-metrics-tls podName:1e94a284-d202-4352-82fa-dd51de2f2c4e nodeName:}" failed. No retries permitted until 2026-07-01 14:39:59.190823094 +0000 UTC m=+36.519395392 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1e94a284-d202-4352-82fa-dd51de2f2c4e-metrics-tls") pod "dns-default-9grqk" (UID: "1e94a284-d202-4352-82fa-dd51de2f2c4e") : secret "dns-default-metrics-tls" not found
Jul 01 14:39:58.544297 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:58.544111    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jw949"]
Jul 01 14:39:58.681488 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:39:58.681447    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf93bdf31_79e7_4d7f_9956_5a5a7e9e5075.slice/crio-72365adc5837d2a34f519713287ad3c798a8458789e6b186c6852fa34f831ce0 WatchSource:0}: Error finding container 72365adc5837d2a34f519713287ad3c798a8458789e6b186c6852fa34f831ce0: Status 404 returned error can't find the container with id 72365adc5837d2a34f519713287ad3c798a8458789e6b186c6852fa34f831ce0
Jul 01 14:39:59.098104 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.098067    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:39:59.098615 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.098067    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:39:59.098615 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.098085    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:39:59.102488 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.102462    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 01 14:39:59.102644 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.102542    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 01 14:39:59.102832 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.102815    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 01 14:39:59.102913 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.102872    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 01 14:39:59.102953 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.102909    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z9fbg\""
Jul 01 14:39:59.103531 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.103518    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lpckn\""
Jul 01 14:39:59.199520 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.199048    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e94a284-d202-4352-82fa-dd51de2f2c4e-metrics-tls\") pod \"dns-default-9grqk\" (UID: \"1e94a284-d202-4352-82fa-dd51de2f2c4e\") " pod="openshift-dns/dns-default-9grqk"
Jul 01 14:39:59.199520 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:59.199136    2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 14:39:59.199520 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.199178    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-registry-tls\") pod \"image-registry-67f9d68f9-xgrmg\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") " pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:39:59.199520 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:59.199237    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e94a284-d202-4352-82fa-dd51de2f2c4e-metrics-tls podName:1e94a284-d202-4352-82fa-dd51de2f2c4e nodeName:}" failed. No retries permitted until 2026-07-01 14:40:01.199205686 +0000 UTC m=+38.527777973 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1e94a284-d202-4352-82fa-dd51de2f2c4e-metrics-tls") pod "dns-default-9grqk" (UID: "1e94a284-d202-4352-82fa-dd51de2f2c4e") : secret "dns-default-metrics-tls" not found
Jul 01 14:39:59.199520 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:59.199307    2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 14:39:59.199520 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:59.199322    2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67f9d68f9-xgrmg: secret "image-registry-tls" not found
Jul 01 14:39:59.199520 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:59.199471    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-registry-tls podName:f4470e2b-27f5-49b2-b979-e72a8e31eddc nodeName:}" failed. No retries permitted until 2026-07-01 14:40:01.199453558 +0000 UTC m=+38.528025858 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-registry-tls") pod "image-registry-67f9d68f9-xgrmg" (UID: "f4470e2b-27f5-49b2-b979-e72a8e31eddc") : secret "image-registry-tls" not found
Jul 01 14:39:59.242121 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.242078    2551 generic.go:356] "Generic (PLEG): container finished" podID="bc612921-fe17-41c3-93ba-2e1197f6f843" containerID="aa295e683bbbc10ac7ebfb29819c581d050200f66c6cef387bfe0c247ab399ff" exitCode=0
Jul 01 14:39:59.242321 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.242154    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nhhr7" event={"ID":"bc612921-fe17-41c3-93ba-2e1197f6f843","Type":"ContainerDied","Data":"aa295e683bbbc10ac7ebfb29819c581d050200f66c6cef387bfe0c247ab399ff"}
Jul 01 14:39:59.243355 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.243336    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jw949" event={"ID":"f93bdf31-79e7-4d7f-9956-5a5a7e9e5075","Type":"ContainerStarted","Data":"72365adc5837d2a34f519713287ad3c798a8458789e6b186c6852fa34f831ce0"}
Jul 01 14:39:59.437998 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.437956    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-f7wph"]
Jul 01 14:39:59.459292 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.459259    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-f7wph"]
Jul 01 14:39:59.459478 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.459398    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-f7wph"
Jul 01 14:39:59.465874 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.465847    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 01 14:39:59.466099 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.466071    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 01 14:39:59.466211 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.466144    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wjs5n\""
Jul 01 14:39:59.466278 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.466238    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 01 14:39:59.466863 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.466839    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 01 14:39:59.603586 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.603502    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ebcec99-5af8-4b1c-9b74-cec8a0712906-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f7wph\" (UID: \"6ebcec99-5af8-4b1c-9b74-cec8a0712906\") " pod="openshift-insights/insights-runtime-extractor-f7wph"
Jul 01 14:39:59.603586 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.603579    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ebcec99-5af8-4b1c-9b74-cec8a0712906-data-volume\") pod \"insights-runtime-extractor-f7wph\" (UID: \"6ebcec99-5af8-4b1c-9b74-cec8a0712906\") " pod="openshift-insights/insights-runtime-extractor-f7wph"
Jul 01 14:39:59.603824 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.603658    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6ebcec99-5af8-4b1c-9b74-cec8a0712906-crio-socket\") pod \"insights-runtime-extractor-f7wph\" (UID: \"6ebcec99-5af8-4b1c-9b74-cec8a0712906\") " pod="openshift-insights/insights-runtime-extractor-f7wph"
Jul 01 14:39:59.603824 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.603700    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjdzg\" (UniqueName: \"kubernetes.io/projected/6ebcec99-5af8-4b1c-9b74-cec8a0712906-kube-api-access-hjdzg\") pod \"insights-runtime-extractor-f7wph\" (UID: \"6ebcec99-5af8-4b1c-9b74-cec8a0712906\") " pod="openshift-insights/insights-runtime-extractor-f7wph"
Jul 01 14:39:59.603824 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.603742    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ebcec99-5af8-4b1c-9b74-cec8a0712906-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f7wph\" (UID: \"6ebcec99-5af8-4b1c-9b74-cec8a0712906\") " pod="openshift-insights/insights-runtime-extractor-f7wph"
Jul 01 14:39:59.705090 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.705057    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ebcec99-5af8-4b1c-9b74-cec8a0712906-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f7wph\" (UID: \"6ebcec99-5af8-4b1c-9b74-cec8a0712906\") " pod="openshift-insights/insights-runtime-extractor-f7wph"
Jul 01 14:39:59.705291 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.705122    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ebcec99-5af8-4b1c-9b74-cec8a0712906-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f7wph\" (UID: \"6ebcec99-5af8-4b1c-9b74-cec8a0712906\") " pod="openshift-insights/insights-runtime-extractor-f7wph"
Jul 01 14:39:59.705291 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.705169    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ebcec99-5af8-4b1c-9b74-cec8a0712906-data-volume\") pod \"insights-runtime-extractor-f7wph\" (UID: \"6ebcec99-5af8-4b1c-9b74-cec8a0712906\") " pod="openshift-insights/insights-runtime-extractor-f7wph"
Jul 01 14:39:59.705291 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:59.705222    2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 14:39:59.705291 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:59.705235    2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 01 14:39:59.705510 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:59.705308    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ebcec99-5af8-4b1c-9b74-cec8a0712906-kube-rbac-proxy-cm podName:6ebcec99-5af8-4b1c-9b74-cec8a0712906 nodeName:}" failed. No retries permitted until 2026-07-01 14:40:00.205287714 +0000 UTC m=+37.533860016 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ebcec99-5af8-4b1c-9b74-cec8a0712906-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f7wph" (UID: "6ebcec99-5af8-4b1c-9b74-cec8a0712906") : configmap "kube-rbac-proxy" not found
Jul 01 14:39:59.705510 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.705339    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6ebcec99-5af8-4b1c-9b74-cec8a0712906-crio-socket\") pod \"insights-runtime-extractor-f7wph\" (UID: \"6ebcec99-5af8-4b1c-9b74-cec8a0712906\") " pod="openshift-insights/insights-runtime-extractor-f7wph"
Jul 01 14:39:59.705510 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.705380    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjdzg\" (UniqueName: \"kubernetes.io/projected/6ebcec99-5af8-4b1c-9b74-cec8a0712906-kube-api-access-hjdzg\") pod \"insights-runtime-extractor-f7wph\" (UID: \"6ebcec99-5af8-4b1c-9b74-cec8a0712906\") " pod="openshift-insights/insights-runtime-extractor-f7wph"
Jul 01 14:39:59.705510 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:39:59.705445    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ebcec99-5af8-4b1c-9b74-cec8a0712906-insights-runtime-extractor-tls podName:6ebcec99-5af8-4b1c-9b74-cec8a0712906 nodeName:}" failed. No retries permitted until 2026-07-01 14:40:00.205420374 +0000 UTC m=+37.533992666 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ebcec99-5af8-4b1c-9b74-cec8a0712906-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f7wph" (UID: "6ebcec99-5af8-4b1c-9b74-cec8a0712906") : secret "insights-runtime-extractor-tls" not found
Jul 01 14:39:59.705705 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.705526    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6ebcec99-5af8-4b1c-9b74-cec8a0712906-crio-socket\") pod \"insights-runtime-extractor-f7wph\" (UID: \"6ebcec99-5af8-4b1c-9b74-cec8a0712906\") " pod="openshift-insights/insights-runtime-extractor-f7wph"
Jul 01 14:39:59.705705 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.705531    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ebcec99-5af8-4b1c-9b74-cec8a0712906-data-volume\") pod \"insights-runtime-extractor-f7wph\" (UID: \"6ebcec99-5af8-4b1c-9b74-cec8a0712906\") " pod="openshift-insights/insights-runtime-extractor-f7wph"
Jul 01 14:39:59.718569 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:39:59.718521    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjdzg\" (UniqueName: \"kubernetes.io/projected/6ebcec99-5af8-4b1c-9b74-cec8a0712906-kube-api-access-hjdzg\") pod \"insights-runtime-extractor-f7wph\" (UID: \"6ebcec99-5af8-4b1c-9b74-cec8a0712906\") " pod="openshift-insights/insights-runtime-extractor-f7wph"
Jul 01 14:40:00.209985 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:00.209949    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ebcec99-5af8-4b1c-9b74-cec8a0712906-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f7wph\" (UID: \"6ebcec99-5af8-4b1c-9b74-cec8a0712906\") " pod="openshift-insights/insights-runtime-extractor-f7wph"
Jul 01 14:40:00.210424 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:00.210022    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ebcec99-5af8-4b1c-9b74-cec8a0712906-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f7wph\" (UID: \"6ebcec99-5af8-4b1c-9b74-cec8a0712906\") " pod="openshift-insights/insights-runtime-extractor-f7wph"
Jul 01 14:40:00.210424 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:00.210186    2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 01 14:40:00.210424 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:00.210244    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ebcec99-5af8-4b1c-9b74-cec8a0712906-kube-rbac-proxy-cm podName:6ebcec99-5af8-4b1c-9b74-cec8a0712906 nodeName:}" failed. No retries permitted until 2026-07-01 14:40:01.21023026 +0000 UTC m=+38.538802544 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ebcec99-5af8-4b1c-9b74-cec8a0712906-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f7wph" (UID: "6ebcec99-5af8-4b1c-9b74-cec8a0712906") : configmap "kube-rbac-proxy" not found
Jul 01 14:40:00.210681 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:00.210590    2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 14:40:00.210681 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:00.210663    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ebcec99-5af8-4b1c-9b74-cec8a0712906-insights-runtime-extractor-tls podName:6ebcec99-5af8-4b1c-9b74-cec8a0712906 nodeName:}" failed. No retries permitted until 2026-07-01 14:40:01.21064978 +0000 UTC m=+38.539222077 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ebcec99-5af8-4b1c-9b74-cec8a0712906-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f7wph" (UID: "6ebcec99-5af8-4b1c-9b74-cec8a0712906") : secret "insights-runtime-extractor-tls" not found
Jul 01 14:40:00.250103 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:00.250055    2551 generic.go:356] "Generic (PLEG): container finished" podID="bc612921-fe17-41c3-93ba-2e1197f6f843" containerID="4d35247a41f6b313461f3b3b975628a1e0ee432e77196db9b42f60328f0baa5b" exitCode=0
Jul 01 14:40:00.250103 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:00.250094    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nhhr7" event={"ID":"bc612921-fe17-41c3-93ba-2e1197f6f843","Type":"ContainerDied","Data":"4d35247a41f6b313461f3b3b975628a1e0ee432e77196db9b42f60328f0baa5b"}
Jul 01 14:40:01.021545 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:01.021368    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rzw6z_4bc04385-5d77-4673-a019-88bb93a250dd/dns-node-resolver/0.log"
Jul 01 14:40:01.220406 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:01.220317    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-registry-tls\") pod \"image-registry-67f9d68f9-xgrmg\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") " pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:40:01.220999 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:01.220408    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ebcec99-5af8-4b1c-9b74-cec8a0712906-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f7wph\" (UID: \"6ebcec99-5af8-4b1c-9b74-cec8a0712906\") " pod="openshift-insights/insights-runtime-extractor-f7wph"
Jul 01 14:40:01.220999 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:01.220465    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ebcec99-5af8-4b1c-9b74-cec8a0712906-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f7wph\" (UID: \"6ebcec99-5af8-4b1c-9b74-cec8a0712906\") " pod="openshift-insights/insights-runtime-extractor-f7wph"
Jul 01 14:40:01.220999 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:01.220484    2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 14:40:01.220999 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:01.220506    2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67f9d68f9-xgrmg: secret "image-registry-tls" not found
Jul 01 14:40:01.220999 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:01.220524    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e94a284-d202-4352-82fa-dd51de2f2c4e-metrics-tls\") pod \"dns-default-9grqk\" (UID: \"1e94a284-d202-4352-82fa-dd51de2f2c4e\") " pod="openshift-dns/dns-default-9grqk"
Jul 01 14:40:01.220999 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:01.220572    2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 14:40:01.220999 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:01.220613    2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 01 14:40:01.220999 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:01.220675    2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 14:40:01.220999 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:01.220579    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-registry-tls podName:f4470e2b-27f5-49b2-b979-e72a8e31eddc nodeName:}" failed. No retries permitted until 2026-07-01 14:40:05.220554737 +0000 UTC m=+42.549127039 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-registry-tls") pod "image-registry-67f9d68f9-xgrmg" (UID: "f4470e2b-27f5-49b2-b979-e72a8e31eddc") : secret "image-registry-tls" not found
Jul 01 14:40:01.220999 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:01.220736    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ebcec99-5af8-4b1c-9b74-cec8a0712906-insights-runtime-extractor-tls podName:6ebcec99-5af8-4b1c-9b74-cec8a0712906 nodeName:}" failed. No retries permitted until 2026-07-01 14:40:03.220718358 +0000 UTC m=+40.549290644 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ebcec99-5af8-4b1c-9b74-cec8a0712906-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f7wph" (UID: "6ebcec99-5af8-4b1c-9b74-cec8a0712906") : secret "insights-runtime-extractor-tls" not found
Jul 01 14:40:01.220999 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:01.220761    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ebcec99-5af8-4b1c-9b74-cec8a0712906-kube-rbac-proxy-cm podName:6ebcec99-5af8-4b1c-9b74-cec8a0712906 nodeName:}" failed. No retries permitted until 2026-07-01 14:40:03.220751333 +0000 UTC m=+40.549323622 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ebcec99-5af8-4b1c-9b74-cec8a0712906-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f7wph" (UID: "6ebcec99-5af8-4b1c-9b74-cec8a0712906") : configmap "kube-rbac-proxy" not found
Jul 01 14:40:01.220999 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:01.220781    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e94a284-d202-4352-82fa-dd51de2f2c4e-metrics-tls podName:1e94a284-d202-4352-82fa-dd51de2f2c4e nodeName:}" failed. No retries permitted until 2026-07-01 14:40:05.220772269 +0000 UTC m=+42.549344557 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1e94a284-d202-4352-82fa-dd51de2f2c4e-metrics-tls") pod "dns-default-9grqk" (UID: "1e94a284-d202-4352-82fa-dd51de2f2c4e") : secret "dns-default-metrics-tls" not found
Jul 01 14:40:01.258673 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:01.258637    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nhhr7" event={"ID":"bc612921-fe17-41c3-93ba-2e1197f6f843","Type":"ContainerStarted","Data":"0596c11f77bb3dcc3ff5468d1867b48b3ce817c70b8683bfd455cb1901ba2a5d"}
Jul 01 14:40:01.297472 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:01.297403    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nhhr7" podStartSLOduration=8.783262008 podStartE2EDuration="38.297379539s" totalImagesPullingTime="29.514117531s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 14:39:23 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:01.297286456 +0000 UTC m=+38.625858764" watchObservedRunningTime="2026-07-01 14:40:01.297379539 +0000 UTC m=+38.625951845"
Jul 01 14:40:02.021049 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:02.021024    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hn47h_49ff50e4-fbc4-4623-a5c5-458c628adef3/node-ca/0.log"
Jul 01 14:40:02.262897 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:02.262861    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jw949" event={"ID":"f93bdf31-79e7-4d7f-9956-5a5a7e9e5075","Type":"ContainerStarted","Data":"e17ac224152a86db756bd0e64aab906c86f28207c248070c4e69b363e92b2a87"}
Jul 01 14:40:02.296398 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:02.296276    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jw949" podStartSLOduration=2.320950127 podStartE2EDuration="5.296252578s" totalImagesPullingTime="2.975302451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:39:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:02.29608963 +0000 UTC m=+39.624661937" watchObservedRunningTime="2026-07-01 14:40:02.296252578 +0000 UTC m=+39.624824885"
Jul 01 14:40:02.621757 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:02.621615    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jw949_f93bdf31-79e7-4d7f-9956-5a5a7e9e5075/serve-healthcheck-canary/0.log"
Jul 01 14:40:03.235595 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:03.235554    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ebcec99-5af8-4b1c-9b74-cec8a0712906-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f7wph\" (UID: \"6ebcec99-5af8-4b1c-9b74-cec8a0712906\") " pod="openshift-insights/insights-runtime-extractor-f7wph"
Jul 01 14:40:03.235787 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:03.235689    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ebcec99-5af8-4b1c-9b74-cec8a0712906-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f7wph\" (UID: \"6ebcec99-5af8-4b1c-9b74-cec8a0712906\") " pod="openshift-insights/insights-runtime-extractor-f7wph"
Jul 01 14:40:03.235787 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:03.235699    2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 01 14:40:03.235787 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:03.235762    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ebcec99-5af8-4b1c-9b74-cec8a0712906-kube-rbac-proxy-cm podName:6ebcec99-5af8-4b1c-9b74-cec8a0712906 nodeName:}" failed. No retries permitted until 2026-07-01 14:40:07.235746676 +0000 UTC m=+44.564318960 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ebcec99-5af8-4b1c-9b74-cec8a0712906-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f7wph" (UID: "6ebcec99-5af8-4b1c-9b74-cec8a0712906") : configmap "kube-rbac-proxy" not found
Jul 01 14:40:03.235918 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:03.235811    2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 14:40:03.235918 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:03.235857    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ebcec99-5af8-4b1c-9b74-cec8a0712906-insights-runtime-extractor-tls podName:6ebcec99-5af8-4b1c-9b74-cec8a0712906 nodeName:}" failed. No retries permitted until 2026-07-01 14:40:07.235846519 +0000 UTC m=+44.564418819 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ebcec99-5af8-4b1c-9b74-cec8a0712906-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f7wph" (UID: "6ebcec99-5af8-4b1c-9b74-cec8a0712906") : secret "insights-runtime-extractor-tls" not found
Jul 01 14:40:05.250309 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:05.250270    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e94a284-d202-4352-82fa-dd51de2f2c4e-metrics-tls\") pod \"dns-default-9grqk\" (UID: \"1e94a284-d202-4352-82fa-dd51de2f2c4e\") " pod="openshift-dns/dns-default-9grqk"
Jul 01 14:40:05.250309 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:05.250318    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-registry-tls\") pod \"image-registry-67f9d68f9-xgrmg\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") " pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:40:05.250785 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:05.250457    2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 14:40:05.250785 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:05.250526    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e94a284-d202-4352-82fa-dd51de2f2c4e-metrics-tls podName:1e94a284-d202-4352-82fa-dd51de2f2c4e nodeName:}" failed. No retries permitted until 2026-07-01 14:40:13.250510184 +0000 UTC m=+50.579082492 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1e94a284-d202-4352-82fa-dd51de2f2c4e-metrics-tls") pod "dns-default-9grqk" (UID: "1e94a284-d202-4352-82fa-dd51de2f2c4e") : secret "dns-default-metrics-tls" not found
Jul 01 14:40:05.250785 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:05.250464    2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 01 14:40:05.250785 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:05.250558    2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67f9d68f9-xgrmg: secret "image-registry-tls" not found
Jul 01 14:40:05.250785 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:05.250604    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-registry-tls podName:f4470e2b-27f5-49b2-b979-e72a8e31eddc nodeName:}" failed. No retries permitted until 2026-07-01 14:40:13.250591767 +0000 UTC m=+50.579164071 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-registry-tls") pod "image-registry-67f9d68f9-xgrmg" (UID: "f4470e2b-27f5-49b2-b979-e72a8e31eddc") : secret "image-registry-tls" not found
Jul 01 14:40:07.265715 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:07.265676    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ebcec99-5af8-4b1c-9b74-cec8a0712906-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f7wph\" (UID: \"6ebcec99-5af8-4b1c-9b74-cec8a0712906\") " pod="openshift-insights/insights-runtime-extractor-f7wph"
Jul 01 14:40:07.266115 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:07.265776    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ebcec99-5af8-4b1c-9b74-cec8a0712906-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f7wph\" (UID: \"6ebcec99-5af8-4b1c-9b74-cec8a0712906\") " pod="openshift-insights/insights-runtime-extractor-f7wph"
Jul 01 14:40:07.266115 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:07.265868    2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 14:40:07.266115 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:07.265922    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ebcec99-5af8-4b1c-9b74-cec8a0712906-insights-runtime-extractor-tls podName:6ebcec99-5af8-4b1c-9b74-cec8a0712906 nodeName:}" failed. No retries permitted until 2026-07-01 14:40:15.265907972 +0000 UTC m=+52.594480256 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ebcec99-5af8-4b1c-9b74-cec8a0712906-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f7wph" (UID: "6ebcec99-5af8-4b1c-9b74-cec8a0712906") : secret "insights-runtime-extractor-tls" not found
Jul 01 14:40:07.266291 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:07.266271    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ebcec99-5af8-4b1c-9b74-cec8a0712906-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f7wph\" (UID: \"6ebcec99-5af8-4b1c-9b74-cec8a0712906\") " pod="openshift-insights/insights-runtime-extractor-f7wph"
Jul 01 14:40:13.314464 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:13.314421    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e94a284-d202-4352-82fa-dd51de2f2c4e-metrics-tls\") pod \"dns-default-9grqk\" (UID: \"1e94a284-d202-4352-82fa-dd51de2f2c4e\") " pod="openshift-dns/dns-default-9grqk"
Jul 01 14:40:13.314464 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:13.314468    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-registry-tls\") pod \"image-registry-67f9d68f9-xgrmg\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") " pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:40:13.317571 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:13.317534    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e94a284-d202-4352-82fa-dd51de2f2c4e-metrics-tls\") pod \"dns-default-9grqk\" (UID: \"1e94a284-d202-4352-82fa-dd51de2f2c4e\") " pod="openshift-dns/dns-default-9grqk"
Jul 01 14:40:13.317794 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:13.317774    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-registry-tls\") pod \"image-registry-67f9d68f9-xgrmg\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") " pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:40:13.420955 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:13.420920    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:40:13.427042 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:13.427021    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9grqk"
Jul 01 14:40:13.561886 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:13.561843    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67f9d68f9-xgrmg"]
Jul 01 14:40:13.566239 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:40:13.566166    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4470e2b_27f5_49b2_b979_e72a8e31eddc.slice/crio-2de130f00a4c46e0a7eabf85037aba5ece2156907b8138b95ac95c1ddb0811fd WatchSource:0}: Error finding container 2de130f00a4c46e0a7eabf85037aba5ece2156907b8138b95ac95c1ddb0811fd: Status 404 returned error can't find the container with id 2de130f00a4c46e0a7eabf85037aba5ece2156907b8138b95ac95c1ddb0811fd
Jul 01 14:40:13.576690 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:13.576642    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9grqk"]
Jul 01 14:40:13.581076 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:40:13.580994    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e94a284_d202_4352_82fa_dd51de2f2c4e.slice/crio-13eeda6dcfc308a80f7d844dc9a7d82075346987e144f8e2a26d99653328cbd7 WatchSource:0}: Error finding container 13eeda6dcfc308a80f7d844dc9a7d82075346987e144f8e2a26d99653328cbd7: Status 404 returned error can't find the container with id 13eeda6dcfc308a80f7d844dc9a7d82075346987e144f8e2a26d99653328cbd7
Jul 01 14:40:14.293406 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:14.293351    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9grqk" event={"ID":"1e94a284-d202-4352-82fa-dd51de2f2c4e","Type":"ContainerStarted","Data":"13eeda6dcfc308a80f7d844dc9a7d82075346987e144f8e2a26d99653328cbd7"}
Jul 01 14:40:14.295131 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:14.295087    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg" event={"ID":"f4470e2b-27f5-49b2-b979-e72a8e31eddc","Type":"ContainerStarted","Data":"1f15241145e42d7c45af0445053418b6b27535dc0192e45eaa434810b041cc93"}
Jul 01 14:40:14.295261 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:14.295152    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg" event={"ID":"f4470e2b-27f5-49b2-b979-e72a8e31eddc","Type":"ContainerStarted","Data":"2de130f00a4c46e0a7eabf85037aba5ece2156907b8138b95ac95c1ddb0811fd"}
Jul 01 14:40:14.295261 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:14.295199    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:40:14.335579 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:14.335521    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg" podStartSLOduration=48.335500676 podStartE2EDuration="48.335500676s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:39:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:14.334868983 +0000 UTC m=+51.663441326" watchObservedRunningTime="2026-07-01 14:40:14.335500676 +0000 UTC m=+51.664072983"
Jul 01 14:40:15.298986 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:15.298949    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9grqk" event={"ID":"1e94a284-d202-4352-82fa-dd51de2f2c4e","Type":"ContainerStarted","Data":"1da0b63598e39c6451338f5ca3b3fd06a0acece1dd08c1f4b5dc5c9e379b88de"}
Jul 01 14:40:15.299168 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:15.299004    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9grqk" event={"ID":"1e94a284-d202-4352-82fa-dd51de2f2c4e","Type":"ContainerStarted","Data":"add9964b90374be32d8d7d5ab5736f7a4293918b2a9e36721e2b907a7da5a860"}
Jul 01 14:40:15.320764 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:15.320651    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9grqk" podStartSLOduration=16.841584143 podStartE2EDuration="18.320617443s" totalImagesPullingTime="1.4790333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:39:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:15.319416729 +0000 UTC m=+52.647989037" watchObservedRunningTime="2026-07-01 14:40:15.320617443 +0000 UTC m=+52.649189727"
Jul 01 14:40:15.334562 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:15.334531    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ebcec99-5af8-4b1c-9b74-cec8a0712906-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f7wph\" (UID: \"6ebcec99-5af8-4b1c-9b74-cec8a0712906\") " pod="openshift-insights/insights-runtime-extractor-f7wph"
Jul 01 14:40:15.336886 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:15.336858    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ebcec99-5af8-4b1c-9b74-cec8a0712906-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f7wph\" (UID: \"6ebcec99-5af8-4b1c-9b74-cec8a0712906\") " pod="openshift-insights/insights-runtime-extractor-f7wph"
Jul 01 14:40:15.390994 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:15.390956    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-f7wph"
Jul 01 14:40:15.519943 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:15.519919    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-f7wph"]
Jul 01 14:40:15.522581 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:40:15.522555    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ebcec99_5af8_4b1c_9b74_cec8a0712906.slice/crio-ede26cb317e1684eac8b5b0481c9ec41ddc53d5d1a10a6377c9df317e65b4da2 WatchSource:0}: Error finding container ede26cb317e1684eac8b5b0481c9ec41ddc53d5d1a10a6377c9df317e65b4da2: Status 404 returned error can't find the container with id ede26cb317e1684eac8b5b0481c9ec41ddc53d5d1a10a6377c9df317e65b4da2
Jul 01 14:40:16.304538 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:16.304501    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f7wph" event={"ID":"6ebcec99-5af8-4b1c-9b74-cec8a0712906","Type":"ContainerStarted","Data":"ede26cb317e1684eac8b5b0481c9ec41ddc53d5d1a10a6377c9df317e65b4da2"}
Jul 01 14:40:16.304767 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:16.304584    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9grqk"
Jul 01 14:40:20.317959 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:20.317929    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f7wph" event={"ID":"6ebcec99-5af8-4b1c-9b74-cec8a0712906","Type":"ContainerStarted","Data":"b6a068ed66d7cb194c67bb172d296173097d6358295f704657539bd98fcbba35"}
Jul 01 14:40:21.323871 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:21.323831    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f7wph" event={"ID":"6ebcec99-5af8-4b1c-9b74-cec8a0712906","Type":"ContainerStarted","Data":"a7f8f8efe6d72986c5a42efc744769c74b4b3ac9b020f7c211bf491b02a08e01"}
Jul 01 14:40:22.230897 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:22.230863    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qnczw"
Jul 01 14:40:23.332502 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:23.332411    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f7wph" event={"ID":"6ebcec99-5af8-4b1c-9b74-cec8a0712906","Type":"ContainerStarted","Data":"376042d6514334ab6ec3a7135165db867c5c353a8330cf727e14e9b767900fad"}
Jul 01 14:40:23.358098 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:23.358044    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-f7wph" podStartSLOduration=17.027455771 podStartE2EDuration="24.358027684s" totalImagesPullingTime="7.330571913s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 14:39:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:23.356796352 +0000 UTC m=+60.685368641" watchObservedRunningTime="2026-07-01 14:40:23.358027684 +0000 UTC m=+60.686599989"
Jul 01 14:40:26.085869 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.085837    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-766b865f6b-s6qtz"]
Jul 01 14:40:26.087827 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.087807    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766b865f6b-s6qtz"
Jul 01 14:40:26.090757 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.090735    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 01 14:40:26.091748 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.091728    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 01 14:40:26.091845 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.091799    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 01 14:40:26.094633 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.094602    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 01 14:40:26.122338 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.122310    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/945a7d7e-d666-4a1d-afbc-10267b1f4dc0-klusterlet-config\") pod \"klusterlet-addon-workmgr-766b865f6b-s6qtz\" (UID: \"945a7d7e-d666-4a1d-afbc-10267b1f4dc0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766b865f6b-s6qtz"
Jul 01 14:40:26.122473 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.122352    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/945a7d7e-d666-4a1d-afbc-10267b1f4dc0-tmp\") pod \"klusterlet-addon-workmgr-766b865f6b-s6qtz\" (UID: \"945a7d7e-d666-4a1d-afbc-10267b1f4dc0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766b865f6b-s6qtz"
Jul 01 14:40:26.122473 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.122377    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hzc2\" (UniqueName: \"kubernetes.io/projected/945a7d7e-d666-4a1d-afbc-10267b1f4dc0-kube-api-access-6hzc2\") pod \"klusterlet-addon-workmgr-766b865f6b-s6qtz\" (UID: \"945a7d7e-d666-4a1d-afbc-10267b1f4dc0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766b865f6b-s6qtz"
Jul 01 14:40:26.130461 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.130435    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-766b865f6b-s6qtz"]
Jul 01 14:40:26.179287 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.179245    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-xmvdl"]
Jul 01 14:40:26.181221 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.181197    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-xmvdl"
Jul 01 14:40:26.186925 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.186902    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-hpb6d\""
Jul 01 14:40:26.187064 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.186982    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 01 14:40:26.190655 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.190610    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-67f9d68f9-xgrmg"]
Jul 01 14:40:26.222957 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.222920    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/945a7d7e-d666-4a1d-afbc-10267b1f4dc0-klusterlet-config\") pod \"klusterlet-addon-workmgr-766b865f6b-s6qtz\" (UID: \"945a7d7e-d666-4a1d-afbc-10267b1f4dc0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766b865f6b-s6qtz"
Jul 01 14:40:26.223132 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.222964    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/945a7d7e-d666-4a1d-afbc-10267b1f4dc0-tmp\") pod \"klusterlet-addon-workmgr-766b865f6b-s6qtz\" (UID: \"945a7d7e-d666-4a1d-afbc-10267b1f4dc0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766b865f6b-s6qtz"
Jul 01 14:40:26.223132 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.223018    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hzc2\" (UniqueName: \"kubernetes.io/projected/945a7d7e-d666-4a1d-afbc-10267b1f4dc0-kube-api-access-6hzc2\") pod \"klusterlet-addon-workmgr-766b865f6b-s6qtz\" (UID: \"945a7d7e-d666-4a1d-afbc-10267b1f4dc0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766b865f6b-s6qtz"
Jul 01 14:40:26.223132 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.223119    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/05d8c9d2-2f1c-4261-8d4a-4203c652007f-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-xmvdl\" (UID: \"05d8c9d2-2f1c-4261-8d4a-4203c652007f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-xmvdl"
Jul 01 14:40:26.223371 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.223352    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/945a7d7e-d666-4a1d-afbc-10267b1f4dc0-tmp\") pod \"klusterlet-addon-workmgr-766b865f6b-s6qtz\" (UID: \"945a7d7e-d666-4a1d-afbc-10267b1f4dc0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766b865f6b-s6qtz"
Jul 01 14:40:26.225491 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.225474    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/945a7d7e-d666-4a1d-afbc-10267b1f4dc0-klusterlet-config\") pod \"klusterlet-addon-workmgr-766b865f6b-s6qtz\" (UID: \"945a7d7e-d666-4a1d-afbc-10267b1f4dc0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766b865f6b-s6qtz"
Jul 01 14:40:26.257674 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.257634    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5499b6fb8f-pm2t5"]
Jul 01 14:40:26.259695 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.259678    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5499b6fb8f-pm2t5"
Jul 01 14:40:26.261681 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.261660    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hzc2\" (UniqueName: \"kubernetes.io/projected/945a7d7e-d666-4a1d-afbc-10267b1f4dc0-kube-api-access-6hzc2\") pod \"klusterlet-addon-workmgr-766b865f6b-s6qtz\" (UID: \"945a7d7e-d666-4a1d-afbc-10267b1f4dc0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766b865f6b-s6qtz"
Jul 01 14:40:26.270138 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.270014    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 01 14:40:26.270138 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.270067    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-blzp4\""
Jul 01 14:40:26.270432 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.270334    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 01 14:40:26.271203 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.271182    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-xmvdl"]
Jul 01 14:40:26.286585 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.286556    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5499b6fb8f-pm2t5"]
Jul 01 14:40:26.309336 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.309309    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9grqk"
Jul 01 14:40:26.312641 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.312576    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5fc59ff786-9vm7x"]
Jul 01 14:40:26.314589 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.314573    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x"
Jul 01 14:40:26.323848 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.323826    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/05d8c9d2-2f1c-4261-8d4a-4203c652007f-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-xmvdl\" (UID: \"05d8c9d2-2f1c-4261-8d4a-4203c652007f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-xmvdl"
Jul 01 14:40:26.323973 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.323873    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrkrf\" (UniqueName: \"kubernetes.io/projected/3bf70ba6-5f5c-4a13-a624-3c5227c62a01-kube-api-access-qrkrf\") pod \"downloads-5499b6fb8f-pm2t5\" (UID: \"3bf70ba6-5f5c-4a13-a624-3c5227c62a01\") " pod="openshift-console/downloads-5499b6fb8f-pm2t5"
Jul 01 14:40:26.326217 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.326196    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/05d8c9d2-2f1c-4261-8d4a-4203c652007f-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-xmvdl\" (UID: \"05d8c9d2-2f1c-4261-8d4a-4203c652007f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-xmvdl"
Jul 01 14:40:26.331571 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.331545    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5fc59ff786-9vm7x"]
Jul 01 14:40:26.397096 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.397008    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766b865f6b-s6qtz"
Jul 01 14:40:26.424833 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.424799    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrkrf\" (UniqueName: \"kubernetes.io/projected/3bf70ba6-5f5c-4a13-a624-3c5227c62a01-kube-api-access-qrkrf\") pod \"downloads-5499b6fb8f-pm2t5\" (UID: \"3bf70ba6-5f5c-4a13-a624-3c5227c62a01\") " pod="openshift-console/downloads-5499b6fb8f-pm2t5"
Jul 01 14:40:26.425038 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.424866    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9632f072-3da7-4b81-9d81-715b0e36890d-image-registry-private-configuration\") pod \"image-registry-5fc59ff786-9vm7x\" (UID: \"9632f072-3da7-4b81-9d81-715b0e36890d\") " pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x"
Jul 01 14:40:26.425038 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.424898    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9632f072-3da7-4b81-9d81-715b0e36890d-installation-pull-secrets\") pod \"image-registry-5fc59ff786-9vm7x\" (UID: \"9632f072-3da7-4b81-9d81-715b0e36890d\") " pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x"
Jul 01 14:40:26.425038 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.424925    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9632f072-3da7-4b81-9d81-715b0e36890d-registry-tls\") pod \"image-registry-5fc59ff786-9vm7x\" (UID: \"9632f072-3da7-4b81-9d81-715b0e36890d\") " pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x"
Jul 01 14:40:26.425038 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.424949    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9632f072-3da7-4b81-9d81-715b0e36890d-ca-trust-extracted\") pod \"image-registry-5fc59ff786-9vm7x\" (UID: \"9632f072-3da7-4b81-9d81-715b0e36890d\") " pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x"
Jul 01 14:40:26.425256 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.425043    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vkhd\" (UniqueName: \"kubernetes.io/projected/9632f072-3da7-4b81-9d81-715b0e36890d-kube-api-access-6vkhd\") pod \"image-registry-5fc59ff786-9vm7x\" (UID: \"9632f072-3da7-4b81-9d81-715b0e36890d\") " pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x"
Jul 01 14:40:26.425256 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.425110    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9632f072-3da7-4b81-9d81-715b0e36890d-registry-certificates\") pod \"image-registry-5fc59ff786-9vm7x\" (UID: \"9632f072-3da7-4b81-9d81-715b0e36890d\") " pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x"
Jul 01 14:40:26.425256 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.425151    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9632f072-3da7-4b81-9d81-715b0e36890d-bound-sa-token\") pod \"image-registry-5fc59ff786-9vm7x\" (UID: \"9632f072-3da7-4b81-9d81-715b0e36890d\") " pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x"
Jul 01 14:40:26.425256 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.425185    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9632f072-3da7-4b81-9d81-715b0e36890d-trusted-ca\") pod \"image-registry-5fc59ff786-9vm7x\" (UID: \"9632f072-3da7-4b81-9d81-715b0e36890d\") " pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x"
Jul 01 14:40:26.440013 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.439984    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrkrf\" (UniqueName: \"kubernetes.io/projected/3bf70ba6-5f5c-4a13-a624-3c5227c62a01-kube-api-access-qrkrf\") pod \"downloads-5499b6fb8f-pm2t5\" (UID: \"3bf70ba6-5f5c-4a13-a624-3c5227c62a01\") " pod="openshift-console/downloads-5499b6fb8f-pm2t5"
Jul 01 14:40:26.490027 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.489985    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-xmvdl"
Jul 01 14:40:26.526558 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.526507    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vkhd\" (UniqueName: \"kubernetes.io/projected/9632f072-3da7-4b81-9d81-715b0e36890d-kube-api-access-6vkhd\") pod \"image-registry-5fc59ff786-9vm7x\" (UID: \"9632f072-3da7-4b81-9d81-715b0e36890d\") " pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x"
Jul 01 14:40:26.526753 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.526594    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9632f072-3da7-4b81-9d81-715b0e36890d-registry-certificates\") pod \"image-registry-5fc59ff786-9vm7x\" (UID: \"9632f072-3da7-4b81-9d81-715b0e36890d\") " pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x"
Jul 01 14:40:26.526753 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.526647    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9632f072-3da7-4b81-9d81-715b0e36890d-bound-sa-token\") pod \"image-registry-5fc59ff786-9vm7x\" (UID: \"9632f072-3da7-4b81-9d81-715b0e36890d\") " pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x"
Jul 01 14:40:26.526753 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.526682    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9632f072-3da7-4b81-9d81-715b0e36890d-trusted-ca\") pod \"image-registry-5fc59ff786-9vm7x\" (UID: \"9632f072-3da7-4b81-9d81-715b0e36890d\") " pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x"
Jul 01 14:40:26.526915 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.526759    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9632f072-3da7-4b81-9d81-715b0e36890d-image-registry-private-configuration\") pod \"image-registry-5fc59ff786-9vm7x\" (UID: \"9632f072-3da7-4b81-9d81-715b0e36890d\") " pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x"
Jul 01 14:40:26.526915 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.526791    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9632f072-3da7-4b81-9d81-715b0e36890d-installation-pull-secrets\") pod \"image-registry-5fc59ff786-9vm7x\" (UID: \"9632f072-3da7-4b81-9d81-715b0e36890d\") " pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x"
Jul 01 14:40:26.526915 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.526820    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9632f072-3da7-4b81-9d81-715b0e36890d-registry-tls\") pod \"image-registry-5fc59ff786-9vm7x\" (UID: \"9632f072-3da7-4b81-9d81-715b0e36890d\") " pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x"
Jul 01 14:40:26.526915 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.526852    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9632f072-3da7-4b81-9d81-715b0e36890d-ca-trust-extracted\") pod \"image-registry-5fc59ff786-9vm7x\" (UID: \"9632f072-3da7-4b81-9d81-715b0e36890d\") " pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x"
Jul 01 14:40:26.527381 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.527352    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9632f072-3da7-4b81-9d81-715b0e36890d-ca-trust-extracted\") pod \"image-registry-5fc59ff786-9vm7x\" (UID: \"9632f072-3da7-4b81-9d81-715b0e36890d\") " pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x"
Jul 01 14:40:26.527612 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.527583    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9632f072-3da7-4b81-9d81-715b0e36890d-registry-certificates\") pod \"image-registry-5fc59ff786-9vm7x\" (UID: \"9632f072-3da7-4b81-9d81-715b0e36890d\") " pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x"
Jul 01 14:40:26.528829 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.528798    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9632f072-3da7-4b81-9d81-715b0e36890d-trusted-ca\") pod \"image-registry-5fc59ff786-9vm7x\" (UID: \"9632f072-3da7-4b81-9d81-715b0e36890d\") " pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x"
Jul 01 14:40:26.530124 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.529861    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9632f072-3da7-4b81-9d81-715b0e36890d-installation-pull-secrets\") pod \"image-registry-5fc59ff786-9vm7x\" (UID: \"9632f072-3da7-4b81-9d81-715b0e36890d\") " pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x"
Jul 01 14:40:26.530229 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.530171    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9632f072-3da7-4b81-9d81-715b0e36890d-registry-tls\") pod \"image-registry-5fc59ff786-9vm7x\" (UID: \"9632f072-3da7-4b81-9d81-715b0e36890d\") " pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x"
Jul 01 14:40:26.530544 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.530528    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9632f072-3da7-4b81-9d81-715b0e36890d-image-registry-private-configuration\") pod \"image-registry-5fc59ff786-9vm7x\" (UID: \"9632f072-3da7-4b81-9d81-715b0e36890d\") " pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x"
Jul 01 14:40:26.533028 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.533006    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-766b865f6b-s6qtz"]
Jul 01 14:40:26.535752 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:40:26.535729    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod945a7d7e_d666_4a1d_afbc_10267b1f4dc0.slice/crio-b0469181dc9cde04ee38e0e4d91627f819560f32a787e06db5abf48268fdd0bc WatchSource:0}: Error finding container b0469181dc9cde04ee38e0e4d91627f819560f32a787e06db5abf48268fdd0bc: Status 404 returned error can't find the container with id b0469181dc9cde04ee38e0e4d91627f819560f32a787e06db5abf48268fdd0bc
Jul 01 14:40:26.538466 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.538303    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9632f072-3da7-4b81-9d81-715b0e36890d-bound-sa-token\") pod \"image-registry-5fc59ff786-9vm7x\" (UID: \"9632f072-3da7-4b81-9d81-715b0e36890d\") " pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x"
Jul 01 14:40:26.539648 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.538820    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vkhd\" (UniqueName: \"kubernetes.io/projected/9632f072-3da7-4b81-9d81-715b0e36890d-kube-api-access-6vkhd\") pod \"image-registry-5fc59ff786-9vm7x\" (UID: \"9632f072-3da7-4b81-9d81-715b0e36890d\") " pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x"
Jul 01 14:40:26.572941 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.572910    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5499b6fb8f-pm2t5"
Jul 01 14:40:26.623446 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.623177    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x"
Jul 01 14:40:26.628180 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.628143    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-xmvdl"]
Jul 01 14:40:26.633101 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:40:26.633076    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05d8c9d2_2f1c_4261_8d4a_4203c652007f.slice/crio-4666ff411e27b4d8be6b9c63602ebe7fd58743727721f8229dc840537dc14f78 WatchSource:0}: Error finding container 4666ff411e27b4d8be6b9c63602ebe7fd58743727721f8229dc840537dc14f78: Status 404 returned error can't find the container with id 4666ff411e27b4d8be6b9c63602ebe7fd58743727721f8229dc840537dc14f78
Jul 01 14:40:26.717243 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.717207    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5499b6fb8f-pm2t5"]
Jul 01 14:40:26.721248 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:40:26.721218    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bf70ba6_5f5c_4a13_a624_3c5227c62a01.slice/crio-cb0897798a9c69f7e7443487af3e71e6b453612ce91545530b6c1224babc3c4b WatchSource:0}: Error finding container cb0897798a9c69f7e7443487af3e71e6b453612ce91545530b6c1224babc3c4b: Status 404 returned error can't find the container with id cb0897798a9c69f7e7443487af3e71e6b453612ce91545530b6c1224babc3c4b
Jul 01 14:40:26.764510 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:26.764473    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5fc59ff786-9vm7x"]
Jul 01 14:40:26.769107 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:40:26.769080    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9632f072_3da7_4b81_9d81_715b0e36890d.slice/crio-43a60f4c6567a88c173cbb29e9a8e79aae2b5de1fa7dca8fab835e7cfa2af21a WatchSource:0}: Error finding container 43a60f4c6567a88c173cbb29e9a8e79aae2b5de1fa7dca8fab835e7cfa2af21a: Status 404 returned error can't find the container with id 43a60f4c6567a88c173cbb29e9a8e79aae2b5de1fa7dca8fab835e7cfa2af21a
Jul 01 14:40:27.348408 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:27.348323    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-xmvdl" event={"ID":"05d8c9d2-2f1c-4261-8d4a-4203c652007f","Type":"ContainerStarted","Data":"4666ff411e27b4d8be6b9c63602ebe7fd58743727721f8229dc840537dc14f78"}
Jul 01 14:40:27.349958 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:27.349921    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766b865f6b-s6qtz" event={"ID":"945a7d7e-d666-4a1d-afbc-10267b1f4dc0","Type":"ContainerStarted","Data":"b0469181dc9cde04ee38e0e4d91627f819560f32a787e06db5abf48268fdd0bc"}
Jul 01 14:40:27.351383 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:27.351345    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5499b6fb8f-pm2t5" event={"ID":"3bf70ba6-5f5c-4a13-a624-3c5227c62a01","Type":"ContainerStarted","Data":"cb0897798a9c69f7e7443487af3e71e6b453612ce91545530b6c1224babc3c4b"}
Jul 01 14:40:27.354209 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:27.354131    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x" event={"ID":"9632f072-3da7-4b81-9d81-715b0e36890d","Type":"ContainerStarted","Data":"54e1ef3674cb0fdfc428c05c242a7cf7a92a6fdf14ae3bdbc6ddd545d2551377"}
Jul 01 14:40:27.354209 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:27.354178    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x" event={"ID":"9632f072-3da7-4b81-9d81-715b0e36890d","Type":"ContainerStarted","Data":"43a60f4c6567a88c173cbb29e9a8e79aae2b5de1fa7dca8fab835e7cfa2af21a"}
Jul 01 14:40:27.354484 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:27.354430    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x"
Jul 01 14:40:28.851519 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:28.851480    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/005968bd-f145-4c81-be95-b3170cafef84-metrics-certs\") pod \"network-metrics-daemon-tprn9\" (UID: \"005968bd-f145-4c81-be95-b3170cafef84\") " pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:40:28.854507 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:28.854484    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 01 14:40:28.864318 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:28.864292    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/005968bd-f145-4c81-be95-b3170cafef84-metrics-certs\") pod \"network-metrics-daemon-tprn9\" (UID: \"005968bd-f145-4c81-be95-b3170cafef84\") " pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:40:28.952725 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:28.952686    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/99ded651-5fce-49b7-8561-d54826fd2690-original-pull-secret\") pod \"global-pull-secret-syncer-2lqtw\" (UID: \"99ded651-5fce-49b7-8561-d54826fd2690\") " pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:40:28.952917 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:28.952775    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsszh\" (UniqueName: \"kubernetes.io/projected/8b8d9348-6896-4fd2-b3c1-8d841219bc43-kube-api-access-tsszh\") pod \"network-check-target-km6cc\" (UID: \"8b8d9348-6896-4fd2-b3c1-8d841219bc43\") " pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:40:28.955374 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:28.955351    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 01 14:40:28.955486 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:28.955397    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 01 14:40:28.965632 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:28.965593    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/99ded651-5fce-49b7-8561-d54826fd2690-original-pull-secret\") pod \"global-pull-secret-syncer-2lqtw\" (UID: \"99ded651-5fce-49b7-8561-d54826fd2690\") " pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:40:28.965828 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:28.965814    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 01 14:40:28.976300 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:28.976273    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsszh\" (UniqueName: \"kubernetes.io/projected/8b8d9348-6896-4fd2-b3c1-8d841219bc43-kube-api-access-tsszh\") pod \"network-check-target-km6cc\" (UID: \"8b8d9348-6896-4fd2-b3c1-8d841219bc43\") " pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:40:29.108836 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:29.108748    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2lqtw"
Jul 01 14:40:29.116460 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:29.116417    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lpckn\""
Jul 01 14:40:29.121197 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:29.121174    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z9fbg\""
Jul 01 14:40:29.123980 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:29.123963    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:40:29.129386 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:29.129364    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tprn9"
Jul 01 14:40:30.398164 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:30.395461    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-xmvdl" event={"ID":"05d8c9d2-2f1c-4261-8d4a-4203c652007f","Type":"ContainerStarted","Data":"cb561677881c4861c26bb4537f8ae4e7f9a41ef47af1b468716885eda915ecfa"}
Jul 01 14:40:30.398164 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:30.396416    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-xmvdl"
Jul 01 14:40:30.398164 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:30.397603    2551 patch_prober.go:28] interesting pod/prometheus-operator-admission-webhook-7ddc8784d4-xmvdl container/prometheus-operator-admission-webhook namespace/openshift-monitoring: Readiness probe status=failure output="Get \"https://10.133.0.11:8443/healthz\": dial tcp 10.133.0.11:8443: connect: connection refused" start-of-body=
Jul 01 14:40:30.398164 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:30.397684    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-xmvdl" podUID="05d8c9d2-2f1c-4261-8d4a-4203c652007f" containerName="prometheus-operator-admission-webhook" probeResult="failure" output="Get \"https://10.133.0.11:8443/healthz\": dial tcp 10.133.0.11:8443: connect: connection refused"
Jul 01 14:40:30.425060 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:30.424876    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x" podStartSLOduration=4.424853442 podStartE2EDuration="4.424853442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:40:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:27.387201281 +0000 UTC m=+64.715773587" watchObservedRunningTime="2026-07-01 14:40:30.424853442 +0000 UTC m=+67.753425746"
Jul 01 14:40:30.425449 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:30.425425    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tprn9"]
Jul 01 14:40:30.428931 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:40:30.428896    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod005968bd_f145_4c81_be95_b3170cafef84.slice/crio-299ce182539c8d6125ace4c74209b21f9dc0906b887483a44d4574123b1ed822 WatchSource:0}: Error finding container 299ce182539c8d6125ace4c74209b21f9dc0906b887483a44d4574123b1ed822: Status 404 returned error can't find the container with id 299ce182539c8d6125ace4c74209b21f9dc0906b887483a44d4574123b1ed822
Jul 01 14:40:30.430856 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:30.430801    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-xmvdl" podStartSLOduration=0.806928224 podStartE2EDuration="4.430780228s" totalImagesPullingTime="3.623852004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:40:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:30.429490516 +0000 UTC m=+67.758062822" watchObservedRunningTime="2026-07-01 14:40:30.430780228 +0000 UTC m=+67.759352535"
Jul 01 14:40:30.446073 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:30.446049    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2lqtw"]
Jul 01 14:40:30.449491 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:40:30.449469    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99ded651_5fce_49b7_8561_d54826fd2690.slice/crio-9cb0f5215eef0c63f788ce3ed91dd11cef28db605e8fd302f2a578c8f16fb9ea WatchSource:0}: Error finding container 9cb0f5215eef0c63f788ce3ed91dd11cef28db605e8fd302f2a578c8f16fb9ea: Status 404 returned error can't find the container with id 9cb0f5215eef0c63f788ce3ed91dd11cef28db605e8fd302f2a578c8f16fb9ea
Jul 01 14:40:30.455561 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:30.455535    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-km6cc"]
Jul 01 14:40:30.458327 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:40:30.458284    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b8d9348_6896_4fd2_b3c1_8d841219bc43.slice/crio-3a376eaf1d1ecc5cc0dad8ad0c1773ebe8fdcbbaa9e506f0c8815608aa1c3568 WatchSource:0}: Error finding container 3a376eaf1d1ecc5cc0dad8ad0c1773ebe8fdcbbaa9e506f0c8815608aa1c3568: Status 404 returned error can't find the container with id 3a376eaf1d1ecc5cc0dad8ad0c1773ebe8fdcbbaa9e506f0c8815608aa1c3568
Jul 01 14:40:31.400983 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.400941    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766b865f6b-s6qtz" event={"ID":"945a7d7e-d666-4a1d-afbc-10267b1f4dc0","Type":"ContainerStarted","Data":"766cc44ee580c85a495b9cf55735777c29d87709832c10a751b7ba9ce3fca5af"}
Jul 01 14:40:31.401459 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.401161    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766b865f6b-s6qtz"
Jul 01 14:40:31.402493 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.402468    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-km6cc" event={"ID":"8b8d9348-6896-4fd2-b3c1-8d841219bc43","Type":"ContainerStarted","Data":"3a376eaf1d1ecc5cc0dad8ad0c1773ebe8fdcbbaa9e506f0c8815608aa1c3568"}
Jul 01 14:40:31.403439 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.403394    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766b865f6b-s6qtz"
Jul 01 14:40:31.404485 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.404408    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2lqtw" event={"ID":"99ded651-5fce-49b7-8561-d54826fd2690","Type":"ContainerStarted","Data":"9cb0f5215eef0c63f788ce3ed91dd11cef28db605e8fd302f2a578c8f16fb9ea"}
Jul 01 14:40:31.405767 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.405744    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tprn9" event={"ID":"005968bd-f145-4c81-be95-b3170cafef84","Type":"ContainerStarted","Data":"299ce182539c8d6125ace4c74209b21f9dc0906b887483a44d4574123b1ed822"}
Jul 01 14:40:31.410979 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.410956    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-xmvdl"
Jul 01 14:40:31.422473 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.422416    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766b865f6b-s6qtz" podStartSLOduration=1.670877735 podStartE2EDuration="5.422399334s" totalImagesPullingTime="3.751521599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:40:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:31.422143065 +0000 UTC m=+68.750715374" watchObservedRunningTime="2026-07-01 14:40:31.422399334 +0000 UTC m=+68.750971638"
Jul 01 14:40:31.737117 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.735953    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d64bb8994-n2p5m"]
Jul 01 14:40:31.738998 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.738967    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d64bb8994-n2p5m"
Jul 01 14:40:31.742286 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.742260    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-x7qb4\""
Jul 01 14:40:31.742650 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.742587    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 01 14:40:31.743309 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.742866    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 01 14:40:31.743309 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.743156    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 01 14:40:31.748882 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.748852    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 01 14:40:31.751580 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.751557    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 01 14:40:31.769076 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.769011    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d64bb8994-n2p5m"]
Jul 01 14:40:31.777172 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.777130    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4j2hf\" (UniqueName: \"kubernetes.io/projected/84990c2d-502d-49a9-aa52-a7e9d9650998-kube-api-access-4j2hf\") pod \"console-6d64bb8994-n2p5m\" (UID: \"84990c2d-502d-49a9-aa52-a7e9d9650998\") " pod="openshift-console/console-6d64bb8994-n2p5m"
Jul 01 14:40:31.777343 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.777183    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84990c2d-502d-49a9-aa52-a7e9d9650998-console-config\") pod \"console-6d64bb8994-n2p5m\" (UID: \"84990c2d-502d-49a9-aa52-a7e9d9650998\") " pod="openshift-console/console-6d64bb8994-n2p5m"
Jul 01 14:40:31.777343 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.777226    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84990c2d-502d-49a9-aa52-a7e9d9650998-console-oauth-config\") pod \"console-6d64bb8994-n2p5m\" (UID: \"84990c2d-502d-49a9-aa52-a7e9d9650998\") " pod="openshift-console/console-6d64bb8994-n2p5m"
Jul 01 14:40:31.777343 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.777267    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84990c2d-502d-49a9-aa52-a7e9d9650998-oauth-serving-cert\") pod \"console-6d64bb8994-n2p5m\" (UID: \"84990c2d-502d-49a9-aa52-a7e9d9650998\") " pod="openshift-console/console-6d64bb8994-n2p5m"
Jul 01 14:40:31.777343 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.777294    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84990c2d-502d-49a9-aa52-a7e9d9650998-console-serving-cert\") pod \"console-6d64bb8994-n2p5m\" (UID: \"84990c2d-502d-49a9-aa52-a7e9d9650998\") " pod="openshift-console/console-6d64bb8994-n2p5m"
Jul 01 14:40:31.777343 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.777320    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84990c2d-502d-49a9-aa52-a7e9d9650998-service-ca\") pod \"console-6d64bb8994-n2p5m\" (UID: \"84990c2d-502d-49a9-aa52-a7e9d9650998\") " pod="openshift-console/console-6d64bb8994-n2p5m"
Jul 01 14:40:31.856045 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.854725    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-99wxn"]
Jul 01 14:40:31.858692 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.857665    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7b7d896497-99wxn"
Jul 01 14:40:31.860963 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.860935    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 01 14:40:31.861463 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.861438    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 01 14:40:31.862009 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.861725    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 01 14:40:31.862009 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.861783    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 01 14:40:31.862009 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.861911    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 01 14:40:31.864662 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.864608    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-w58jf\""
Jul 01 14:40:31.872123 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.872099    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-99wxn"]
Jul 01 14:40:31.883580 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.883532    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84990c2d-502d-49a9-aa52-a7e9d9650998-console-config\") pod \"console-6d64bb8994-n2p5m\" (UID: \"84990c2d-502d-49a9-aa52-a7e9d9650998\") " pod="openshift-console/console-6d64bb8994-n2p5m"
Jul 01 14:40:31.883714 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.883592    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84990c2d-502d-49a9-aa52-a7e9d9650998-console-oauth-config\") pod \"console-6d64bb8994-n2p5m\" (UID: \"84990c2d-502d-49a9-aa52-a7e9d9650998\") " pod="openshift-console/console-6d64bb8994-n2p5m"
Jul 01 14:40:31.885306 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.884331    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84990c2d-502d-49a9-aa52-a7e9d9650998-console-config\") pod \"console-6d64bb8994-n2p5m\" (UID: \"84990c2d-502d-49a9-aa52-a7e9d9650998\") " pod="openshift-console/console-6d64bb8994-n2p5m"
Jul 01 14:40:31.885306 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.883659    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84990c2d-502d-49a9-aa52-a7e9d9650998-oauth-serving-cert\") pod \"console-6d64bb8994-n2p5m\" (UID: \"84990c2d-502d-49a9-aa52-a7e9d9650998\") " pod="openshift-console/console-6d64bb8994-n2p5m"
Jul 01 14:40:31.885306 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.884407    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84990c2d-502d-49a9-aa52-a7e9d9650998-console-serving-cert\") pod \"console-6d64bb8994-n2p5m\" (UID: \"84990c2d-502d-49a9-aa52-a7e9d9650998\") " pod="openshift-console/console-6d64bb8994-n2p5m"
Jul 01 14:40:31.885306 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.884432    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84990c2d-502d-49a9-aa52-a7e9d9650998-service-ca\") pod \"console-6d64bb8994-n2p5m\" (UID: \"84990c2d-502d-49a9-aa52-a7e9d9650998\") " pod="openshift-console/console-6d64bb8994-n2p5m"
Jul 01 14:40:31.885306 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.884459    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84990c2d-502d-49a9-aa52-a7e9d9650998-oauth-serving-cert\") pod \"console-6d64bb8994-n2p5m\" (UID: \"84990c2d-502d-49a9-aa52-a7e9d9650998\") " pod="openshift-console/console-6d64bb8994-n2p5m"
Jul 01 14:40:31.885306 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.884511    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4j2hf\" (UniqueName: \"kubernetes.io/projected/84990c2d-502d-49a9-aa52-a7e9d9650998-kube-api-access-4j2hf\") pod \"console-6d64bb8994-n2p5m\" (UID: \"84990c2d-502d-49a9-aa52-a7e9d9650998\") " pod="openshift-console/console-6d64bb8994-n2p5m"
Jul 01 14:40:31.886071 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.886031    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84990c2d-502d-49a9-aa52-a7e9d9650998-service-ca\") pod \"console-6d64bb8994-n2p5m\" (UID: \"84990c2d-502d-49a9-aa52-a7e9d9650998\") " pod="openshift-console/console-6d64bb8994-n2p5m"
Jul 01 14:40:31.888328 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.888284    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84990c2d-502d-49a9-aa52-a7e9d9650998-console-oauth-config\") pod \"console-6d64bb8994-n2p5m\" (UID: \"84990c2d-502d-49a9-aa52-a7e9d9650998\") " pod="openshift-console/console-6d64bb8994-n2p5m"
Jul 01 14:40:31.888328 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.888294    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84990c2d-502d-49a9-aa52-a7e9d9650998-console-serving-cert\") pod \"console-6d64bb8994-n2p5m\" (UID: \"84990c2d-502d-49a9-aa52-a7e9d9650998\") " pod="openshift-console/console-6d64bb8994-n2p5m"
Jul 01 14:40:31.894182 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.894158    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4j2hf\" (UniqueName: \"kubernetes.io/projected/84990c2d-502d-49a9-aa52-a7e9d9650998-kube-api-access-4j2hf\") pod \"console-6d64bb8994-n2p5m\" (UID: \"84990c2d-502d-49a9-aa52-a7e9d9650998\") " pod="openshift-console/console-6d64bb8994-n2p5m"
Jul 01 14:40:31.985809 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.985772    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/00f36b77-b452-442a-917e-c49f5902ccce-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-99wxn\" (UID: \"00f36b77-b452-442a-917e-c49f5902ccce\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-99wxn"
Jul 01 14:40:31.986004 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.985838    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00f36b77-b452-442a-917e-c49f5902ccce-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-99wxn\" (UID: \"00f36b77-b452-442a-917e-c49f5902ccce\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-99wxn"
Jul 01 14:40:31.986004 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.985876    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/00f36b77-b452-442a-917e-c49f5902ccce-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-99wxn\" (UID: \"00f36b77-b452-442a-917e-c49f5902ccce\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-99wxn"
Jul 01 14:40:31.986108 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:31.985993    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx256\" (UniqueName: \"kubernetes.io/projected/00f36b77-b452-442a-917e-c49f5902ccce-kube-api-access-qx256\") pod \"prometheus-operator-7b7d896497-99wxn\" (UID: \"00f36b77-b452-442a-917e-c49f5902ccce\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-99wxn"
Jul 01 14:40:32.053904 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:32.053863    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d64bb8994-n2p5m"
Jul 01 14:40:32.086432 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:32.086392    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00f36b77-b452-442a-917e-c49f5902ccce-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-99wxn\" (UID: \"00f36b77-b452-442a-917e-c49f5902ccce\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-99wxn"
Jul 01 14:40:32.086636 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:32.086478    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/00f36b77-b452-442a-917e-c49f5902ccce-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-99wxn\" (UID: \"00f36b77-b452-442a-917e-c49f5902ccce\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-99wxn"
Jul 01 14:40:32.086636 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:32.086572    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx256\" (UniqueName: \"kubernetes.io/projected/00f36b77-b452-442a-917e-c49f5902ccce-kube-api-access-qx256\") pod \"prometheus-operator-7b7d896497-99wxn\" (UID: \"00f36b77-b452-442a-917e-c49f5902ccce\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-99wxn"
Jul 01 14:40:32.086757 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:32.086641    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/00f36b77-b452-442a-917e-c49f5902ccce-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-99wxn\" (UID: \"00f36b77-b452-442a-917e-c49f5902ccce\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-99wxn"
Jul 01 14:40:32.087192 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:32.087166    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00f36b77-b452-442a-917e-c49f5902ccce-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-99wxn\" (UID: \"00f36b77-b452-442a-917e-c49f5902ccce\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-99wxn"
Jul 01 14:40:32.087305 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:32.087204    2551 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 01 14:40:32.087305 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:32.087264    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00f36b77-b452-442a-917e-c49f5902ccce-prometheus-operator-tls podName:00f36b77-b452-442a-917e-c49f5902ccce nodeName:}" failed. No retries permitted until 2026-07-01 14:40:32.587245509 +0000 UTC m=+69.915817803 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/00f36b77-b452-442a-917e-c49f5902ccce-prometheus-operator-tls") pod "prometheus-operator-7b7d896497-99wxn" (UID: "00f36b77-b452-442a-917e-c49f5902ccce") : secret "prometheus-operator-tls" not found
Jul 01 14:40:32.089046 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:32.089020    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/00f36b77-b452-442a-917e-c49f5902ccce-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-99wxn\" (UID: \"00f36b77-b452-442a-917e-c49f5902ccce\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-99wxn"
Jul 01 14:40:32.099968 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:32.099939    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx256\" (UniqueName: \"kubernetes.io/projected/00f36b77-b452-442a-917e-c49f5902ccce-kube-api-access-qx256\") pod \"prometheus-operator-7b7d896497-99wxn\" (UID: \"00f36b77-b452-442a-917e-c49f5902ccce\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-99wxn"
Jul 01 14:40:32.225433 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:32.225401    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d64bb8994-n2p5m"]
Jul 01 14:40:32.228083 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:40:32.228048    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84990c2d_502d_49a9_aa52_a7e9d9650998.slice/crio-880e32b8660351eb74e66febefcb62bb665cc0583fd3546cc4773f268a5fcf54 WatchSource:0}: Error finding container 880e32b8660351eb74e66febefcb62bb665cc0583fd3546cc4773f268a5fcf54: Status 404 returned error can't find the container with id 880e32b8660351eb74e66febefcb62bb665cc0583fd3546cc4773f268a5fcf54
Jul 01 14:40:32.411064 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:32.410967    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tprn9" event={"ID":"005968bd-f145-4c81-be95-b3170cafef84","Type":"ContainerStarted","Data":"067989d0b6787a66fd36ff0881aef3c94dab50481af0799c134ad903fa92173d"}
Jul 01 14:40:32.411064 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:32.411036    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tprn9" event={"ID":"005968bd-f145-4c81-be95-b3170cafef84","Type":"ContainerStarted","Data":"9f2e12b5129742adb67f0bbd6a33ae3809f242a4485d8b51076b06edc2712454"}
Jul 01 14:40:32.412265 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:32.412239    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d64bb8994-n2p5m" event={"ID":"84990c2d-502d-49a9-aa52-a7e9d9650998","Type":"ContainerStarted","Data":"880e32b8660351eb74e66febefcb62bb665cc0583fd3546cc4773f268a5fcf54"}
Jul 01 14:40:32.430953 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:32.430896    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-tprn9" podStartSLOduration=68.049201413 podStartE2EDuration="1m9.430874339s" totalImagesPullingTime="1.381672926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:39:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:32.430792256 +0000 UTC m=+69.759364563" watchObservedRunningTime="2026-07-01 14:40:32.430874339 +0000 UTC m=+69.759446644"
Jul 01 14:40:32.591355 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:32.591312    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/00f36b77-b452-442a-917e-c49f5902ccce-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-99wxn\" (UID: \"00f36b77-b452-442a-917e-c49f5902ccce\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-99wxn"
Jul 01 14:40:32.591572 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:32.591485    2551 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 01 14:40:32.591572 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:32.591549    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00f36b77-b452-442a-917e-c49f5902ccce-prometheus-operator-tls podName:00f36b77-b452-442a-917e-c49f5902ccce nodeName:}" failed. No retries permitted until 2026-07-01 14:40:33.591533009 +0000 UTC m=+70.920105304 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/00f36b77-b452-442a-917e-c49f5902ccce-prometheus-operator-tls") pod "prometheus-operator-7b7d896497-99wxn" (UID: "00f36b77-b452-442a-917e-c49f5902ccce") : secret "prometheus-operator-tls" not found
Jul 01 14:40:33.601507 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:33.601275    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/00f36b77-b452-442a-917e-c49f5902ccce-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-99wxn\" (UID: \"00f36b77-b452-442a-917e-c49f5902ccce\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-99wxn"
Jul 01 14:40:33.606455 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:33.606372    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/00f36b77-b452-442a-917e-c49f5902ccce-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-99wxn\" (UID: \"00f36b77-b452-442a-917e-c49f5902ccce\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-99wxn"
Jul 01 14:40:33.676270 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:33.676230    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7b7d896497-99wxn"
Jul 01 14:40:36.198840 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:36.198536    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:40:36.714904 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:36.714821    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-99wxn"]
Jul 01 14:40:36.718926 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:40:36.718893    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00f36b77_b452_442a_917e_c49f5902ccce.slice/crio-05d67c13962e3fedd639c8700c7faa70af9feaf1496f037b061ae596f1f2120e WatchSource:0}: Error finding container 05d67c13962e3fedd639c8700c7faa70af9feaf1496f037b061ae596f1f2120e: Status 404 returned error can't find the container with id 05d67c13962e3fedd639c8700c7faa70af9feaf1496f037b061ae596f1f2120e
Jul 01 14:40:37.437352 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:37.437309    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d64bb8994-n2p5m" event={"ID":"84990c2d-502d-49a9-aa52-a7e9d9650998","Type":"ContainerStarted","Data":"b4c17a93ece8410076ebcf8ee1968eec34343f43282791898166e38c572d9271"}
Jul 01 14:40:37.439018 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:37.438984    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2lqtw" event={"ID":"99ded651-5fce-49b7-8561-d54826fd2690","Type":"ContainerStarted","Data":"ef209dc3466bef4766f190a5129217e5420419a8a4e1fad00597d60e782000d3"}
Jul 01 14:40:37.440287 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:37.440261    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-99wxn" event={"ID":"00f36b77-b452-442a-917e-c49f5902ccce","Type":"ContainerStarted","Data":"05d67c13962e3fedd639c8700c7faa70af9feaf1496f037b061ae596f1f2120e"}
Jul 01 14:40:37.524454 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:37.524388    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d64bb8994-n2p5m" podStartSLOduration=2.238491433 podStartE2EDuration="6.524363581s" totalImagesPullingTime="4.285872148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:40:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:37.523561334 +0000 UTC m=+74.852133642" watchObservedRunningTime="2026-07-01 14:40:37.524363581 +0000 UTC m=+74.852935889"
Jul 01 14:40:37.553952 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:37.553881    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2lqtw" podStartSLOduration=68.48760675 podStartE2EDuration="1m14.553856495s" totalImagesPullingTime="6.066249745s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 14:39:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:37.552787625 +0000 UTC m=+74.881359933" watchObservedRunningTime="2026-07-01 14:40:37.553856495 +0000 UTC m=+74.882428801"
Jul 01 14:40:38.449004 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:38.448841    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-km6cc" event={"ID":"8b8d9348-6896-4fd2-b3c1-8d841219bc43","Type":"ContainerStarted","Data":"c4eb1e252f0b40549b29aa6565793385ca5c51756d7062990073bd553c7395b9"}
Jul 01 14:40:38.449533 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:38.449511    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:40:38.483247 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:38.483153    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-km6cc" podStartSLOduration=67.889797583 podStartE2EDuration="1m15.483131493s" totalImagesPullingTime="7.59333391s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:39:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:38.48132972 +0000 UTC m=+75.809902025" watchObservedRunningTime="2026-07-01 14:40:38.483131493 +0000 UTC m=+75.811703800"
Jul 01 14:40:39.455307 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:39.455214    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-99wxn" event={"ID":"00f36b77-b452-442a-917e-c49f5902ccce","Type":"ContainerStarted","Data":"08bccc76809b84945e7cde7be45da96aa7ed19b9ba18cb85edf0d9e8f09ec2ec"}
Jul 01 14:40:39.455307 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:39.455283    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-99wxn" event={"ID":"00f36b77-b452-442a-917e-c49f5902ccce","Type":"ContainerStarted","Data":"df4c10013444d301142c7f62946fb827b941d996b2827327a522d352fcb195df"}
Jul 01 14:40:39.476126 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:39.476046    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7b7d896497-99wxn" podStartSLOduration=6.219485765 podStartE2EDuration="8.476023133s" totalImagesPullingTime="2.256537368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:40:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:39.474885553 +0000 UTC m=+76.803457871" watchObservedRunningTime="2026-07-01 14:40:39.476023133 +0000 UTC m=+76.804595440"
Jul 01 14:40:40.251399 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:40.251358    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c75897b6f-974mb"]
Jul 01 14:40:40.253707 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:40.253670    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c75897b6f-974mb"
Jul 01 14:40:40.262786 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:40.262758    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 01 14:40:40.270868 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:40.270838    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c75897b6f-974mb"]
Jul 01 14:40:40.360417 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:40.360383    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-trusted-ca-bundle\") pod \"console-6c75897b6f-974mb\" (UID: \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\") " pod="openshift-console/console-6c75897b6f-974mb"
Jul 01 14:40:40.360591 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:40.360427    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-console-oauth-config\") pod \"console-6c75897b6f-974mb\" (UID: \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\") " pod="openshift-console/console-6c75897b6f-974mb"
Jul 01 14:40:40.360591 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:40.360448    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kfr6\" (UniqueName: \"kubernetes.io/projected/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-kube-api-access-4kfr6\") pod \"console-6c75897b6f-974mb\" (UID: \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\") " pod="openshift-console/console-6c75897b6f-974mb"
Jul 01 14:40:40.360591 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:40.360465    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-oauth-serving-cert\") pod \"console-6c75897b6f-974mb\" (UID: \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\") " pod="openshift-console/console-6c75897b6f-974mb"
Jul 01 14:40:40.360591 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:40.360512    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-console-serving-cert\") pod \"console-6c75897b6f-974mb\" (UID: \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\") " pod="openshift-console/console-6c75897b6f-974mb"
Jul 01 14:40:40.360591 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:40.360533    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-console-config\") pod \"console-6c75897b6f-974mb\" (UID: \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\") " pod="openshift-console/console-6c75897b6f-974mb"
Jul 01 14:40:40.360811 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:40.360607    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-service-ca\") pod \"console-6c75897b6f-974mb\" (UID: \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\") " pod="openshift-console/console-6c75897b6f-974mb"
Jul 01 14:40:40.461917 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:40.461880    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-console-oauth-config\") pod \"console-6c75897b6f-974mb\" (UID: \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\") " pod="openshift-console/console-6c75897b6f-974mb"
Jul 01 14:40:40.461917 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:40.461916    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kfr6\" (UniqueName: \"kubernetes.io/projected/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-kube-api-access-4kfr6\") pod \"console-6c75897b6f-974mb\" (UID: \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\") " pod="openshift-console/console-6c75897b6f-974mb"
Jul 01 14:40:40.462427 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:40.461936    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-oauth-serving-cert\") pod \"console-6c75897b6f-974mb\" (UID: \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\") " pod="openshift-console/console-6c75897b6f-974mb"
Jul 01 14:40:40.462427 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:40.462139    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-console-serving-cert\") pod \"console-6c75897b6f-974mb\" (UID: \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\") " pod="openshift-console/console-6c75897b6f-974mb"
Jul 01 14:40:40.462427 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:40.462174    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-console-config\") pod \"console-6c75897b6f-974mb\" (UID: \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\") " pod="openshift-console/console-6c75897b6f-974mb"
Jul 01 14:40:40.462427 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:40.462264    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-service-ca\") pod \"console-6c75897b6f-974mb\" (UID: \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\") " pod="openshift-console/console-6c75897b6f-974mb"
Jul 01 14:40:40.462427 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:40.462314    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-trusted-ca-bundle\") pod \"console-6c75897b6f-974mb\" (UID: \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\") " pod="openshift-console/console-6c75897b6f-974mb"
Jul 01 14:40:40.462754 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:40.462711    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-oauth-serving-cert\") pod \"console-6c75897b6f-974mb\" (UID: \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\") " pod="openshift-console/console-6c75897b6f-974mb"
Jul 01 14:40:40.463048 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:40.463018    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-console-config\") pod \"console-6c75897b6f-974mb\" (UID: \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\") " pod="openshift-console/console-6c75897b6f-974mb"
Jul 01 14:40:40.463133 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:40.463085    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-service-ca\") pod \"console-6c75897b6f-974mb\" (UID: \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\") " pod="openshift-console/console-6c75897b6f-974mb"
Jul 01 14:40:40.463737 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:40.463716    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-trusted-ca-bundle\") pod \"console-6c75897b6f-974mb\" (UID: \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\") " pod="openshift-console/console-6c75897b6f-974mb"
Jul 01 14:40:40.464431 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:40.464413    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-console-oauth-config\") pod \"console-6c75897b6f-974mb\" (UID: \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\") " pod="openshift-console/console-6c75897b6f-974mb"
Jul 01 14:40:40.464668 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:40.464643    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-console-serving-cert\") pod \"console-6c75897b6f-974mb\" (UID: \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\") " pod="openshift-console/console-6c75897b6f-974mb"
Jul 01 14:40:40.472156 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:40.472125    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kfr6\" (UniqueName: \"kubernetes.io/projected/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-kube-api-access-4kfr6\") pod \"console-6c75897b6f-974mb\" (UID: \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\") " pod="openshift-console/console-6c75897b6f-974mb"
Jul 01 14:40:40.565331 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:40.565234    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c75897b6f-974mb"
Jul 01 14:40:40.709642 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:40.709228    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c75897b6f-974mb"]
Jul 01 14:40:41.309635 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.309592    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-svkc6"]
Jul 01 14:40:41.311973 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.311955    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-845b954455-svkc6"
Jul 01 14:40:41.316292 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.316267    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 01 14:40:41.317316 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.317286    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-92vqn\""
Jul 01 14:40:41.317316 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.317315    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 01 14:40:41.323018 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.322992    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 01 14:40:41.342094 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.342057    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-b2qcb"]
Jul 01 14:40:41.344594 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.344568    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b2qcb"
Jul 01 14:40:41.346521 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.346495    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-svkc6"]
Jul 01 14:40:41.348093 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.348076    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tpkpl\""
Jul 01 14:40:41.348438 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.348419    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 01 14:40:41.348531 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.348419    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 01 14:40:41.348531 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.348425    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 01 14:40:41.467189 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.467149    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c75897b6f-974mb" event={"ID":"1c2198a5-7d68-401c-8889-6ab86dd9ac4f","Type":"ContainerStarted","Data":"615b60efc8481195d0b7d1c6751e54c541596779224b12a3e9417802d41a2885"}
Jul 01 14:40:41.467650 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.467213    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c75897b6f-974mb" event={"ID":"1c2198a5-7d68-401c-8889-6ab86dd9ac4f","Type":"ContainerStarted","Data":"bce04a49355b3c405184d0d013fecda4b4ee9d3e0c82794d7a36d3085fe9f9eb"}
Jul 01 14:40:41.471157 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.471126    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/652dda43-a27b-4fbf-b04f-ca9a0267861a-sys\") pod \"node-exporter-b2qcb\" (UID: \"652dda43-a27b-4fbf-b04f-ca9a0267861a\") " pod="openshift-monitoring/node-exporter-b2qcb"
Jul 01 14:40:41.471289 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.471176    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/652dda43-a27b-4fbf-b04f-ca9a0267861a-node-exporter-textfile\") pod \"node-exporter-b2qcb\" (UID: \"652dda43-a27b-4fbf-b04f-ca9a0267861a\") " pod="openshift-monitoring/node-exporter-b2qcb"
Jul 01 14:40:41.471289 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.471213    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-svkc6\" (UID: \"b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1\") " pod="openshift-monitoring/kube-state-metrics-845b954455-svkc6"
Jul 01 14:40:41.471289 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.471244    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-svkc6\" (UID: \"b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1\") " pod="openshift-monitoring/kube-state-metrics-845b954455-svkc6"
Jul 01 14:40:41.471289 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.471271    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-svkc6\" (UID: \"b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1\") " pod="openshift-monitoring/kube-state-metrics-845b954455-svkc6"
Jul 01 14:40:41.471494 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.471299    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/652dda43-a27b-4fbf-b04f-ca9a0267861a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b2qcb\" (UID: \"652dda43-a27b-4fbf-b04f-ca9a0267861a\") " pod="openshift-monitoring/node-exporter-b2qcb"
Jul 01 14:40:41.471494 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.471327    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/652dda43-a27b-4fbf-b04f-ca9a0267861a-node-exporter-tls\") pod \"node-exporter-b2qcb\" (UID: \"652dda43-a27b-4fbf-b04f-ca9a0267861a\") " pod="openshift-monitoring/node-exporter-b2qcb"
Jul 01 14:40:41.471494 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.471356    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/652dda43-a27b-4fbf-b04f-ca9a0267861a-node-exporter-wtmp\") pod \"node-exporter-b2qcb\" (UID: \"652dda43-a27b-4fbf-b04f-ca9a0267861a\") " pod="openshift-monitoring/node-exporter-b2qcb"
Jul 01 14:40:41.471494 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.471477    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1-metrics-client-ca\") pod \"kube-state-metrics-845b954455-svkc6\" (UID: \"b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1\") " pod="openshift-monitoring/kube-state-metrics-845b954455-svkc6"
Jul 01 14:40:41.471714 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.471511    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kq9n\" (UniqueName: \"kubernetes.io/projected/b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1-kube-api-access-4kq9n\") pod \"kube-state-metrics-845b954455-svkc6\" (UID: \"b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1\") " pod="openshift-monitoring/kube-state-metrics-845b954455-svkc6"
Jul 01 14:40:41.471714 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.471543    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/652dda43-a27b-4fbf-b04f-ca9a0267861a-metrics-client-ca\") pod \"node-exporter-b2qcb\" (UID: \"652dda43-a27b-4fbf-b04f-ca9a0267861a\") " pod="openshift-monitoring/node-exporter-b2qcb"
Jul 01 14:40:41.471714 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.471579    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/652dda43-a27b-4fbf-b04f-ca9a0267861a-root\") pod \"node-exporter-b2qcb\" (UID: \"652dda43-a27b-4fbf-b04f-ca9a0267861a\") " pod="openshift-monitoring/node-exporter-b2qcb"
Jul 01 14:40:41.471714 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.471612    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/652dda43-a27b-4fbf-b04f-ca9a0267861a-node-exporter-accelerators-collector-config\") pod \"node-exporter-b2qcb\" (UID: \"652dda43-a27b-4fbf-b04f-ca9a0267861a\") " pod="openshift-monitoring/node-exporter-b2qcb"
Jul 01 14:40:41.471714 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.471656    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwwnv\" (UniqueName: \"kubernetes.io/projected/652dda43-a27b-4fbf-b04f-ca9a0267861a-kube-api-access-fwwnv\") pod \"node-exporter-b2qcb\" (UID: \"652dda43-a27b-4fbf-b04f-ca9a0267861a\") " pod="openshift-monitoring/node-exporter-b2qcb"
Jul 01 14:40:41.471714 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.471698    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-svkc6\" (UID: \"b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1\") " pod="openshift-monitoring/kube-state-metrics-845b954455-svkc6"
Jul 01 14:40:41.510130 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.510075    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c75897b6f-974mb" podStartSLOduration=1.510055882 podStartE2EDuration="1.510055882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:40:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:41.508775739 +0000 UTC m=+78.837348045" watchObservedRunningTime="2026-07-01 14:40:41.510055882 +0000 UTC m=+78.838628188"
Jul 01 14:40:41.572160 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.572061    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-svkc6\" (UID: \"b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1\") " pod="openshift-monitoring/kube-state-metrics-845b954455-svkc6"
Jul 01 14:40:41.572160 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.572107    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-svkc6\" (UID: \"b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1\") " pod="openshift-monitoring/kube-state-metrics-845b954455-svkc6"
Jul 01 14:40:41.572160 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.572134    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/652dda43-a27b-4fbf-b04f-ca9a0267861a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b2qcb\" (UID: \"652dda43-a27b-4fbf-b04f-ca9a0267861a\") " pod="openshift-monitoring/node-exporter-b2qcb"
Jul 01 14:40:41.572160 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.572163    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/652dda43-a27b-4fbf-b04f-ca9a0267861a-node-exporter-tls\") pod \"node-exporter-b2qcb\" (UID: \"652dda43-a27b-4fbf-b04f-ca9a0267861a\") " pod="openshift-monitoring/node-exporter-b2qcb"
Jul 01 14:40:41.572546 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.572199    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/652dda43-a27b-4fbf-b04f-ca9a0267861a-node-exporter-wtmp\") pod \"node-exporter-b2qcb\" (UID: \"652dda43-a27b-4fbf-b04f-ca9a0267861a\") " pod="openshift-monitoring/node-exporter-b2qcb"
Jul 01 14:40:41.572546 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.572266    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1-metrics-client-ca\") pod \"kube-state-metrics-845b954455-svkc6\" (UID: \"b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1\") " pod="openshift-monitoring/kube-state-metrics-845b954455-svkc6"
Jul 01 14:40:41.572546 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.572292    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kq9n\" (UniqueName: \"kubernetes.io/projected/b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1-kube-api-access-4kq9n\") pod \"kube-state-metrics-845b954455-svkc6\" (UID: \"b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1\") " pod="openshift-monitoring/kube-state-metrics-845b954455-svkc6"
Jul 01 14:40:41.572546 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.572318    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/652dda43-a27b-4fbf-b04f-ca9a0267861a-metrics-client-ca\") pod \"node-exporter-b2qcb\" (UID: \"652dda43-a27b-4fbf-b04f-ca9a0267861a\") " pod="openshift-monitoring/node-exporter-b2qcb"
Jul 01 14:40:41.572546 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:41.572334    2551 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 01 14:40:41.572546 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.572358    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/652dda43-a27b-4fbf-b04f-ca9a0267861a-root\") pod \"node-exporter-b2qcb\" (UID: \"652dda43-a27b-4fbf-b04f-ca9a0267861a\") " pod="openshift-monitoring/node-exporter-b2qcb"
Jul 01 14:40:41.572546 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:41.572414    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/652dda43-a27b-4fbf-b04f-ca9a0267861a-node-exporter-tls podName:652dda43-a27b-4fbf-b04f-ca9a0267861a nodeName:}" failed. No retries permitted until 2026-07-01 14:40:42.072390685 +0000 UTC m=+79.400962974 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/652dda43-a27b-4fbf-b04f-ca9a0267861a-node-exporter-tls") pod "node-exporter-b2qcb" (UID: "652dda43-a27b-4fbf-b04f-ca9a0267861a") : secret "node-exporter-tls" not found
Jul 01 14:40:41.572546 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.572472    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/652dda43-a27b-4fbf-b04f-ca9a0267861a-node-exporter-accelerators-collector-config\") pod \"node-exporter-b2qcb\" (UID: \"652dda43-a27b-4fbf-b04f-ca9a0267861a\") " pod="openshift-monitoring/node-exporter-b2qcb"
Jul 01 14:40:41.572546 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.572503    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwwnv\" (UniqueName: \"kubernetes.io/projected/652dda43-a27b-4fbf-b04f-ca9a0267861a-kube-api-access-fwwnv\") pod \"node-exporter-b2qcb\" (UID: \"652dda43-a27b-4fbf-b04f-ca9a0267861a\") " pod="openshift-monitoring/node-exporter-b2qcb"
Jul 01 14:40:41.573028 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.572570    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-svkc6\" (UID: \"b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1\") " pod="openshift-monitoring/kube-state-metrics-845b954455-svkc6"
Jul 01 14:40:41.573028 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.572601    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-svkc6\" (UID: \"b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1\") " pod="openshift-monitoring/kube-state-metrics-845b954455-svkc6"
Jul 01 14:40:41.573028 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.572693    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/652dda43-a27b-4fbf-b04f-ca9a0267861a-sys\") pod \"node-exporter-b2qcb\" (UID: \"652dda43-a27b-4fbf-b04f-ca9a0267861a\") " pod="openshift-monitoring/node-exporter-b2qcb"
Jul 01 14:40:41.573028 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.572734    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/652dda43-a27b-4fbf-b04f-ca9a0267861a-node-exporter-textfile\") pod \"node-exporter-b2qcb\" (UID: \"652dda43-a27b-4fbf-b04f-ca9a0267861a\") " pod="openshift-monitoring/node-exporter-b2qcb"
Jul 01 14:40:41.573028 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.572770    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-svkc6\" (UID: \"b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1\") " pod="openshift-monitoring/kube-state-metrics-845b954455-svkc6"
Jul 01 14:40:41.573028 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.572956    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/652dda43-a27b-4fbf-b04f-ca9a0267861a-node-exporter-wtmp\") pod \"node-exporter-b2qcb\" (UID: \"652dda43-a27b-4fbf-b04f-ca9a0267861a\") " pod="openshift-monitoring/node-exporter-b2qcb"
Jul 01 14:40:41.573028 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.572998    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/652dda43-a27b-4fbf-b04f-ca9a0267861a-sys\") pod \"node-exporter-b2qcb\" (UID: \"652dda43-a27b-4fbf-b04f-ca9a0267861a\") " pod="openshift-monitoring/node-exporter-b2qcb"
Jul 01 14:40:41.573497 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.573251    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/652dda43-a27b-4fbf-b04f-ca9a0267861a-node-exporter-textfile\") pod \"node-exporter-b2qcb\" (UID: \"652dda43-a27b-4fbf-b04f-ca9a0267861a\") " pod="openshift-monitoring/node-exporter-b2qcb"
Jul 01 14:40:41.573497 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.573448    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/652dda43-a27b-4fbf-b04f-ca9a0267861a-node-exporter-accelerators-collector-config\") pod \"node-exporter-b2qcb\" (UID: \"652dda43-a27b-4fbf-b04f-ca9a0267861a\") " pod="openshift-monitoring/node-exporter-b2qcb"
Jul 01 14:40:41.573889 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.573716    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/652dda43-a27b-4fbf-b04f-ca9a0267861a-root\") pod \"node-exporter-b2qcb\" (UID: \"652dda43-a27b-4fbf-b04f-ca9a0267861a\") " pod="openshift-monitoring/node-exporter-b2qcb"
Jul 01 14:40:41.573889 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:41.573721    2551 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 01 14:40:41.573889 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:41.573824    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1-kube-state-metrics-tls podName:b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1 nodeName:}" failed. No retries permitted until 2026-07-01 14:40:42.073803204 +0000 UTC m=+79.402375503 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1-kube-state-metrics-tls") pod "kube-state-metrics-845b954455-svkc6" (UID: "b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1") : secret "kube-state-metrics-tls" not found
Jul 01 14:40:41.574175 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.574004    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/652dda43-a27b-4fbf-b04f-ca9a0267861a-metrics-client-ca\") pod \"node-exporter-b2qcb\" (UID: \"652dda43-a27b-4fbf-b04f-ca9a0267861a\") " pod="openshift-monitoring/node-exporter-b2qcb"
Jul 01 14:40:41.574175 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.574131    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-svkc6\" (UID: \"b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1\") " pod="openshift-monitoring/kube-state-metrics-845b954455-svkc6"
Jul 01 14:40:41.574290 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.574194    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1-metrics-client-ca\") pod \"kube-state-metrics-845b954455-svkc6\" (UID: \"b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1\") " pod="openshift-monitoring/kube-state-metrics-845b954455-svkc6"
Jul 01 14:40:41.575319 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.575295    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/652dda43-a27b-4fbf-b04f-ca9a0267861a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b2qcb\" (UID: \"652dda43-a27b-4fbf-b04f-ca9a0267861a\") " pod="openshift-monitoring/node-exporter-b2qcb"
Jul 01 14:40:41.576363 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.576340    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-svkc6\" (UID: \"b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1\") " pod="openshift-monitoring/kube-state-metrics-845b954455-svkc6"
Jul 01 14:40:41.591517 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.591483    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwwnv\" (UniqueName: \"kubernetes.io/projected/652dda43-a27b-4fbf-b04f-ca9a0267861a-kube-api-access-fwwnv\") pod \"node-exporter-b2qcb\" (UID: \"652dda43-a27b-4fbf-b04f-ca9a0267861a\") " pod="openshift-monitoring/node-exporter-b2qcb"
Jul 01 14:40:41.595407 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:41.595382    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kq9n\" (UniqueName: \"kubernetes.io/projected/b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1-kube-api-access-4kq9n\") pod \"kube-state-metrics-845b954455-svkc6\" (UID: \"b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1\") " pod="openshift-monitoring/kube-state-metrics-845b954455-svkc6"
Jul 01 14:40:42.054963 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:42.054922    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d64bb8994-n2p5m"
Jul 01 14:40:42.055167 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:42.054990    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d64bb8994-n2p5m"
Jul 01 14:40:42.061090 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:42.061055    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d64bb8994-n2p5m"
Jul 01 14:40:42.078925 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:42.078889    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-svkc6\" (UID: \"b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1\") " pod="openshift-monitoring/kube-state-metrics-845b954455-svkc6"
Jul 01 14:40:42.079122 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:42.078945    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/652dda43-a27b-4fbf-b04f-ca9a0267861a-node-exporter-tls\") pod \"node-exporter-b2qcb\" (UID: \"652dda43-a27b-4fbf-b04f-ca9a0267861a\") " pod="openshift-monitoring/node-exporter-b2qcb"
Jul 01 14:40:42.081524 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:42.081495    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/652dda43-a27b-4fbf-b04f-ca9a0267861a-node-exporter-tls\") pod \"node-exporter-b2qcb\" (UID: \"652dda43-a27b-4fbf-b04f-ca9a0267861a\") " pod="openshift-monitoring/node-exporter-b2qcb"
Jul 01 14:40:42.081832 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:42.081805    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-svkc6\" (UID: \"b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1\") " pod="openshift-monitoring/kube-state-metrics-845b954455-svkc6"
Jul 01 14:40:42.221562 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:42.221525    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-845b954455-svkc6"
Jul 01 14:40:42.254767 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:42.254730    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b2qcb"
Jul 01 14:40:42.266090 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:40:42.266058    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod652dda43_a27b_4fbf_b04f_ca9a0267861a.slice/crio-d4b2dbee86e39b2ad67c1d7fc9c878ec332c75d20239b921691017181c21515a WatchSource:0}: Error finding container d4b2dbee86e39b2ad67c1d7fc9c878ec332c75d20239b921691017181c21515a: Status 404 returned error can't find the container with id d4b2dbee86e39b2ad67c1d7fc9c878ec332c75d20239b921691017181c21515a
Jul 01 14:40:42.390976 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:42.390950    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-svkc6"]
Jul 01 14:40:42.393409 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:40:42.393375    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb996e3e7_4810_4b61_88e8_d8e7e7c7f7a1.slice/crio-e3203b7e4bc7e167c70627c44a024b3578d537333180c00dc048405dd09f2a1f WatchSource:0}: Error finding container e3203b7e4bc7e167c70627c44a024b3578d537333180c00dc048405dd09f2a1f: Status 404 returned error can't find the container with id e3203b7e4bc7e167c70627c44a024b3578d537333180c00dc048405dd09f2a1f
Jul 01 14:40:42.473936 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:42.473901    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b2qcb" event={"ID":"652dda43-a27b-4fbf-b04f-ca9a0267861a","Type":"ContainerStarted","Data":"d4b2dbee86e39b2ad67c1d7fc9c878ec332c75d20239b921691017181c21515a"}
Jul 01 14:40:42.475274 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:42.475243    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-svkc6" event={"ID":"b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1","Type":"ContainerStarted","Data":"e3203b7e4bc7e167c70627c44a024b3578d537333180c00dc048405dd09f2a1f"}
Jul 01 14:40:42.479582 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:42.479554    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d64bb8994-n2p5m"
Jul 01 14:40:44.487374 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:44.487330    2551 generic.go:356] "Generic (PLEG): container finished" podID="652dda43-a27b-4fbf-b04f-ca9a0267861a" containerID="5ee1d43ccc2b14f6a207701ad5b126f360f72e2c72967413a0a4480d75ed339b" exitCode=0
Jul 01 14:40:44.487843 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:44.487385    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b2qcb" event={"ID":"652dda43-a27b-4fbf-b04f-ca9a0267861a","Type":"ContainerDied","Data":"5ee1d43ccc2b14f6a207701ad5b126f360f72e2c72967413a0a4480d75ed339b"}
Jul 01 14:40:46.142713 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:46.142672    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c75897b6f-974mb"]
Jul 01 14:40:48.362517 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:48.362486    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5fc59ff786-9vm7x"
Jul 01 14:40:50.565744 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:50.565693    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c75897b6f-974mb"
Jul 01 14:40:51.209951 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:51.209900    2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg" podUID="f4470e2b-27f5-49b2-b979-e72a8e31eddc" containerName="registry" containerID="cri-o://1f15241145e42d7c45af0445053418b6b27535dc0192e45eaa434810b041cc93" gracePeriod=30
Jul 01 14:40:51.821108 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:51.821082    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:40:51.874645 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:51.874599    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5z5rf\" (UniqueName: \"kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-kube-api-access-5z5rf\") pod \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") "
Jul 01 14:40:51.874845 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:51.874669    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f4470e2b-27f5-49b2-b979-e72a8e31eddc-installation-pull-secrets\") pod \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") "
Jul 01 14:40:51.874845 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:51.874697    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-bound-sa-token\") pod \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") "
Jul 01 14:40:51.874845 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:51.874726    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-registry-tls\") pod \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") "
Jul 01 14:40:51.875012 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:51.874847    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f4470e2b-27f5-49b2-b979-e72a8e31eddc-registry-certificates\") pod \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") "
Jul 01 14:40:51.875012 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:51.874895    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f4470e2b-27f5-49b2-b979-e72a8e31eddc-trusted-ca\") pod \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") "
Jul 01 14:40:51.875012 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:51.874946    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f4470e2b-27f5-49b2-b979-e72a8e31eddc-ca-trust-extracted\") pod \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") "
Jul 01 14:40:51.875012 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:51.874988    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f4470e2b-27f5-49b2-b979-e72a8e31eddc-image-registry-private-configuration\") pod \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\" (UID: \"f4470e2b-27f5-49b2-b979-e72a8e31eddc\") "
Jul 01 14:40:51.876137 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:51.875821    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4470e2b-27f5-49b2-b979-e72a8e31eddc-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f4470e2b-27f5-49b2-b979-e72a8e31eddc" (UID: "f4470e2b-27f5-49b2-b979-e72a8e31eddc"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:40:51.876488 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:51.876428    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4470e2b-27f5-49b2-b979-e72a8e31eddc-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f4470e2b-27f5-49b2-b979-e72a8e31eddc" (UID: "f4470e2b-27f5-49b2-b979-e72a8e31eddc"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:40:51.878435 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:51.878343    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-kube-api-access-5z5rf" (OuterVolumeSpecName: "kube-api-access-5z5rf") pod "f4470e2b-27f5-49b2-b979-e72a8e31eddc" (UID: "f4470e2b-27f5-49b2-b979-e72a8e31eddc"). InnerVolumeSpecName "kube-api-access-5z5rf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 14:40:51.878985 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:51.878953    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4470e2b-27f5-49b2-b979-e72a8e31eddc-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f4470e2b-27f5-49b2-b979-e72a8e31eddc" (UID: "f4470e2b-27f5-49b2-b979-e72a8e31eddc"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 14:40:51.879179 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:51.879142    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f4470e2b-27f5-49b2-b979-e72a8e31eddc" (UID: "f4470e2b-27f5-49b2-b979-e72a8e31eddc"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 14:40:51.879179 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:51.879164    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4470e2b-27f5-49b2-b979-e72a8e31eddc-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f4470e2b-27f5-49b2-b979-e72a8e31eddc" (UID: "f4470e2b-27f5-49b2-b979-e72a8e31eddc"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 14:40:51.879694 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:51.879666    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f4470e2b-27f5-49b2-b979-e72a8e31eddc" (UID: "f4470e2b-27f5-49b2-b979-e72a8e31eddc"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 14:40:51.884680 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:51.884649    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4470e2b-27f5-49b2-b979-e72a8e31eddc-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f4470e2b-27f5-49b2-b979-e72a8e31eddc" (UID: "f4470e2b-27f5-49b2-b979-e72a8e31eddc"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 14:40:51.975824 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:51.975750    2551 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-registry-tls\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:40:51.975824 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:51.975789    2551 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f4470e2b-27f5-49b2-b979-e72a8e31eddc-registry-certificates\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:40:51.975824 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:51.975803    2551 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f4470e2b-27f5-49b2-b979-e72a8e31eddc-trusted-ca\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:40:51.975824 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:51.975817    2551 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f4470e2b-27f5-49b2-b979-e72a8e31eddc-ca-trust-extracted\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:40:51.975824 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:51.975832    2551 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f4470e2b-27f5-49b2-b979-e72a8e31eddc-image-registry-private-configuration\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:40:51.976285 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:51.975848    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5z5rf\" (UniqueName: \"kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-kube-api-access-5z5rf\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:40:51.976285 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:51.975861    2551 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f4470e2b-27f5-49b2-b979-e72a8e31eddc-installation-pull-secrets\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:40:51.976285 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:51.975873    2551 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f4470e2b-27f5-49b2-b979-e72a8e31eddc-bound-sa-token\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:40:52.534695 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:52.534660    2551 generic.go:356] "Generic (PLEG): container finished" podID="f4470e2b-27f5-49b2-b979-e72a8e31eddc" containerID="1f15241145e42d7c45af0445053418b6b27535dc0192e45eaa434810b041cc93" exitCode=0
Jul 01 14:40:52.535012 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:52.534729    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg" event={"ID":"f4470e2b-27f5-49b2-b979-e72a8e31eddc","Type":"ContainerDied","Data":"1f15241145e42d7c45af0445053418b6b27535dc0192e45eaa434810b041cc93"}
Jul 01 14:40:52.535172 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:52.534769    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg"
Jul 01 14:40:52.535172 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:52.535052    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67f9d68f9-xgrmg" event={"ID":"f4470e2b-27f5-49b2-b979-e72a8e31eddc","Type":"ContainerDied","Data":"2de130f00a4c46e0a7eabf85037aba5ece2156907b8138b95ac95c1ddb0811fd"}
Jul 01 14:40:52.538489 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:52.537872    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-svkc6" event={"ID":"b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1","Type":"ContainerStarted","Data":"945ff65ad836e94043bfdd0b357f2e0c44aa75e669b3130312a9792bee785412"}
Jul 01 14:40:52.538489 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:52.537943    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-svkc6" event={"ID":"b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1","Type":"ContainerStarted","Data":"ebd988a8f5713120a56ee8e179695af7ae6680e30b3225e5a404949c7151ea88"}
Jul 01 14:40:52.538489 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:52.537974    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-svkc6" event={"ID":"b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1","Type":"ContainerStarted","Data":"abd2575a5a278bcc86551cb0d4eb3ccf1d13a022cd19e0ec8689a5a7c7c88630"}
Jul 01 14:40:52.542454 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:52.542431    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5499b6fb8f-pm2t5" event={"ID":"3bf70ba6-5f5c-4a13-a624-3c5227c62a01","Type":"ContainerStarted","Data":"18cdfcacd193ed2632eb9218d1588c8b07ce457f59cc233e68bfb37ebfe3acd4"}
Jul 01 14:40:52.542873 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:52.542831    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5499b6fb8f-pm2t5"
Jul 01 14:40:52.545343 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:52.545322    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b2qcb" event={"ID":"652dda43-a27b-4fbf-b04f-ca9a0267861a","Type":"ContainerStarted","Data":"eb834b13653a66262f8d580691be3f1a4010a2daa212ed59376840417f776ca8"}
Jul 01 14:40:52.545449 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:52.545366    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b2qcb" event={"ID":"652dda43-a27b-4fbf-b04f-ca9a0267861a","Type":"ContainerStarted","Data":"85eb31e325869febaa7a9734e46a4976c69ee929f0e0765f05df86733974282e"}
Jul 01 14:40:52.550768 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:52.550694    2551 scope.go:117] "RemoveContainer" containerID="1f15241145e42d7c45af0445053418b6b27535dc0192e45eaa434810b041cc93"
Jul 01 14:40:52.551385 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:52.551369    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5499b6fb8f-pm2t5"
Jul 01 14:40:52.563575 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:52.563509    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-845b954455-svkc6" podStartSLOduration=2.140176307 podStartE2EDuration="11.563485252s" totalImagesPullingTime="9.423308945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:40:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:52.562616977 +0000 UTC m=+89.891189337" watchObservedRunningTime="2026-07-01 14:40:52.563485252 +0000 UTC m=+89.892057561"
Jul 01 14:40:52.568964 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:52.568830    2551 scope.go:117] "RemoveContainer" containerID="1f15241145e42d7c45af0445053418b6b27535dc0192e45eaa434810b041cc93"
Jul 01 14:40:52.569286 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:40:52.569246    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f15241145e42d7c45af0445053418b6b27535dc0192e45eaa434810b041cc93\": container with ID starting with 1f15241145e42d7c45af0445053418b6b27535dc0192e45eaa434810b041cc93 not found: ID does not exist" containerID="1f15241145e42d7c45af0445053418b6b27535dc0192e45eaa434810b041cc93"
Jul 01 14:40:52.569432 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:52.569388    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f15241145e42d7c45af0445053418b6b27535dc0192e45eaa434810b041cc93"} err="failed to get container status \"1f15241145e42d7c45af0445053418b6b27535dc0192e45eaa434810b041cc93\": rpc error: code = NotFound desc = could not find container \"1f15241145e42d7c45af0445053418b6b27535dc0192e45eaa434810b041cc93\": container with ID starting with 1f15241145e42d7c45af0445053418b6b27535dc0192e45eaa434810b041cc93 not found: ID does not exist"
Jul 01 14:40:52.592367 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:52.592299    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-b2qcb" podStartSLOduration=10.106828476 podStartE2EDuration="11.59227663s" totalImagesPullingTime="1.398994966s" totalInitContainerRuntime="86.453188ms" isStatefulPod=true podCreationTimestamp="2026-07-01 14:40:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:52.591159998 +0000 UTC m=+89.919732307" watchObservedRunningTime="2026-07-01 14:40:52.59227663 +0000 UTC m=+89.920848937"
Jul 01 14:40:52.609158 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:52.609117    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-67f9d68f9-xgrmg"]
Jul 01 14:40:52.617760 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:52.617723    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-67f9d68f9-xgrmg"]
Jul 01 14:40:52.639540 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:52.639461    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5499b6fb8f-pm2t5" podStartSLOduration=1.495508385 podStartE2EDuration="26.639437505s" totalImagesPullingTime="25.14392912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:40:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:40:52.638865371 +0000 UTC m=+89.967437700" watchObservedRunningTime="2026-07-01 14:40:52.639437505 +0000 UTC m=+89.968009813"
Jul 01 14:40:53.112200 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:53.112164    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4470e2b-27f5-49b2-b979-e72a8e31eddc" path="/var/lib/kubelet/pods/f4470e2b-27f5-49b2-b979-e72a8e31eddc/volumes"
Jul 01 14:40:57.444453 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:40:57.444417    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d64bb8994-n2p5m"]
Jul 01 14:41:10.461768 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:10.461734    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-km6cc"
Jul 01 14:41:11.166765 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.166697    2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6c75897b6f-974mb" podUID="1c2198a5-7d68-401c-8889-6ab86dd9ac4f" containerName="console" containerID="cri-o://615b60efc8481195d0b7d1c6751e54c541596779224b12a3e9417802d41a2885" gracePeriod=15
Jul 01 14:41:11.439184 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.439113    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c75897b6f-974mb"
Jul 01 14:41:11.550804 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.550764    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-trusted-ca-bundle\") pod \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\" (UID: \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\") "
Jul 01 14:41:11.551304 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.550832    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-console-serving-cert\") pod \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\" (UID: \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\") "
Jul 01 14:41:11.551304 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.550866    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-console-oauth-config\") pod \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\" (UID: \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\") "
Jul 01 14:41:11.551304 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.550912    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4kfr6\" (UniqueName: \"kubernetes.io/projected/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-kube-api-access-4kfr6\") pod \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\" (UID: \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\") "
Jul 01 14:41:11.551304 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.550945    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-oauth-serving-cert\") pod \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\" (UID: \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\") "
Jul 01 14:41:11.551304 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.550969    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-console-config\") pod \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\" (UID: \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\") "
Jul 01 14:41:11.551304 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.551005    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-service-ca\") pod \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\" (UID: \"1c2198a5-7d68-401c-8889-6ab86dd9ac4f\") "
Jul 01 14:41:11.551603 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.551332    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1c2198a5-7d68-401c-8889-6ab86dd9ac4f" (UID: "1c2198a5-7d68-401c-8889-6ab86dd9ac4f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:41:11.551603 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.551476    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1c2198a5-7d68-401c-8889-6ab86dd9ac4f" (UID: "1c2198a5-7d68-401c-8889-6ab86dd9ac4f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:41:11.551603 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.551484    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-console-config" (OuterVolumeSpecName: "console-config") pod "1c2198a5-7d68-401c-8889-6ab86dd9ac4f" (UID: "1c2198a5-7d68-401c-8889-6ab86dd9ac4f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:41:11.551603 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.551485    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-service-ca" (OuterVolumeSpecName: "service-ca") pod "1c2198a5-7d68-401c-8889-6ab86dd9ac4f" (UID: "1c2198a5-7d68-401c-8889-6ab86dd9ac4f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:41:11.553211 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.553183    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-kube-api-access-4kfr6" (OuterVolumeSpecName: "kube-api-access-4kfr6") pod "1c2198a5-7d68-401c-8889-6ab86dd9ac4f" (UID: "1c2198a5-7d68-401c-8889-6ab86dd9ac4f"). InnerVolumeSpecName "kube-api-access-4kfr6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 14:41:11.553475 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.553447    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1c2198a5-7d68-401c-8889-6ab86dd9ac4f" (UID: "1c2198a5-7d68-401c-8889-6ab86dd9ac4f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 14:41:11.553549 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.553465    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1c2198a5-7d68-401c-8889-6ab86dd9ac4f" (UID: "1c2198a5-7d68-401c-8889-6ab86dd9ac4f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 14:41:11.619734 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.619697    2551 generic.go:356] "Generic (PLEG): container finished" podID="1c2198a5-7d68-401c-8889-6ab86dd9ac4f" containerID="615b60efc8481195d0b7d1c6751e54c541596779224b12a3e9417802d41a2885" exitCode=0
Jul 01 14:41:11.619929 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.619746    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c75897b6f-974mb"
Jul 01 14:41:11.619929 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.619782    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c75897b6f-974mb" event={"ID":"1c2198a5-7d68-401c-8889-6ab86dd9ac4f","Type":"ContainerDied","Data":"615b60efc8481195d0b7d1c6751e54c541596779224b12a3e9417802d41a2885"}
Jul 01 14:41:11.619929 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.619836    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c75897b6f-974mb" event={"ID":"1c2198a5-7d68-401c-8889-6ab86dd9ac4f","Type":"ContainerDied","Data":"bce04a49355b3c405184d0d013fecda4b4ee9d3e0c82794d7a36d3085fe9f9eb"}
Jul 01 14:41:11.619929 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.619858    2551 scope.go:117] "RemoveContainer" containerID="615b60efc8481195d0b7d1c6751e54c541596779224b12a3e9417802d41a2885"
Jul 01 14:41:11.628118 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.628096    2551 scope.go:117] "RemoveContainer" containerID="615b60efc8481195d0b7d1c6751e54c541596779224b12a3e9417802d41a2885"
Jul 01 14:41:11.628368 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:41:11.628353    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"615b60efc8481195d0b7d1c6751e54c541596779224b12a3e9417802d41a2885\": container with ID starting with 615b60efc8481195d0b7d1c6751e54c541596779224b12a3e9417802d41a2885 not found: ID does not exist" containerID="615b60efc8481195d0b7d1c6751e54c541596779224b12a3e9417802d41a2885"
Jul 01 14:41:11.628411 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.628380    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"615b60efc8481195d0b7d1c6751e54c541596779224b12a3e9417802d41a2885"} err="failed to get container status \"615b60efc8481195d0b7d1c6751e54c541596779224b12a3e9417802d41a2885\": rpc error: code = NotFound desc = could not find container \"615b60efc8481195d0b7d1c6751e54c541596779224b12a3e9417802d41a2885\": container with ID starting with 615b60efc8481195d0b7d1c6751e54c541596779224b12a3e9417802d41a2885 not found: ID does not exist"
Jul 01 14:41:11.641686 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.641653    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c75897b6f-974mb"]
Jul 01 14:41:11.648439 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.648406    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c75897b6f-974mb"]
Jul 01 14:41:11.651631 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.651596    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kfr6\" (UniqueName: \"kubernetes.io/projected/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-kube-api-access-4kfr6\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:41:11.651718 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.651641    2551 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-oauth-serving-cert\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:41:11.651718 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.651652    2551 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-console-config\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:41:11.651718 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.651661    2551 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-service-ca\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:41:11.651718 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.651669    2551 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-trusted-ca-bundle\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:41:11.651718 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.651677    2551 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-console-serving-cert\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:41:11.651718 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:11.651685    2551 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c2198a5-7d68-401c-8889-6ab86dd9ac4f-console-oauth-config\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:41:13.103165 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:13.103129    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c2198a5-7d68-401c-8889-6ab86dd9ac4f" path="/var/lib/kubelet/pods/1c2198a5-7d68-401c-8889-6ab86dd9ac4f/volumes"
Jul 01 14:41:22.469581 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:22.469503    2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6d64bb8994-n2p5m" podUID="84990c2d-502d-49a9-aa52-a7e9d9650998" containerName="console" containerID="cri-o://b4c17a93ece8410076ebcf8ee1968eec34343f43282791898166e38c572d9271" gracePeriod=15
Jul 01 14:41:22.476409 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:22.476369    2551 patch_prober.go:28] interesting pod/console-6d64bb8994-n2p5m container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.133.0.14:8443/health\": dial tcp 10.133.0.14:8443: connect: connection refused" start-of-body=
Jul 01 14:41:22.476538 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:22.476428    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-6d64bb8994-n2p5m" podUID="84990c2d-502d-49a9-aa52-a7e9d9650998" containerName="console" probeResult="failure" output="Get \"https://10.133.0.14:8443/health\": dial tcp 10.133.0.14:8443: connect: connection refused"
Jul 01 14:41:22.666874 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:22.666824    2551 generic.go:356] "Generic (PLEG): container finished" podID="84990c2d-502d-49a9-aa52-a7e9d9650998" containerID="b4c17a93ece8410076ebcf8ee1968eec34343f43282791898166e38c572d9271" exitCode=0
Jul 01 14:41:22.667062 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:22.666876    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d64bb8994-n2p5m" event={"ID":"84990c2d-502d-49a9-aa52-a7e9d9650998","Type":"ContainerDied","Data":"b4c17a93ece8410076ebcf8ee1968eec34343f43282791898166e38c572d9271"}
Jul 01 14:41:22.753063 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:22.753026    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d64bb8994-n2p5m"
Jul 01 14:41:22.836534 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:22.836496    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84990c2d-502d-49a9-aa52-a7e9d9650998-console-oauth-config\") pod \"84990c2d-502d-49a9-aa52-a7e9d9650998\" (UID: \"84990c2d-502d-49a9-aa52-a7e9d9650998\") "
Jul 01 14:41:22.836739 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:22.836563    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84990c2d-502d-49a9-aa52-a7e9d9650998-console-config\") pod \"84990c2d-502d-49a9-aa52-a7e9d9650998\" (UID: \"84990c2d-502d-49a9-aa52-a7e9d9650998\") "
Jul 01 14:41:22.836739 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:22.836587    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4j2hf\" (UniqueName: \"kubernetes.io/projected/84990c2d-502d-49a9-aa52-a7e9d9650998-kube-api-access-4j2hf\") pod \"84990c2d-502d-49a9-aa52-a7e9d9650998\" (UID: \"84990c2d-502d-49a9-aa52-a7e9d9650998\") "
Jul 01 14:41:22.836739 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:22.836608    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84990c2d-502d-49a9-aa52-a7e9d9650998-oauth-serving-cert\") pod \"84990c2d-502d-49a9-aa52-a7e9d9650998\" (UID: \"84990c2d-502d-49a9-aa52-a7e9d9650998\") "
Jul 01 14:41:22.836869 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:22.836746    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84990c2d-502d-49a9-aa52-a7e9d9650998-console-serving-cert\") pod \"84990c2d-502d-49a9-aa52-a7e9d9650998\" (UID: \"84990c2d-502d-49a9-aa52-a7e9d9650998\") "
Jul 01 14:41:22.836869 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:22.836799    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84990c2d-502d-49a9-aa52-a7e9d9650998-service-ca\") pod \"84990c2d-502d-49a9-aa52-a7e9d9650998\" (UID: \"84990c2d-502d-49a9-aa52-a7e9d9650998\") "
Jul 01 14:41:22.837100 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:22.837037    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84990c2d-502d-49a9-aa52-a7e9d9650998-console-config" (OuterVolumeSpecName: "console-config") pod "84990c2d-502d-49a9-aa52-a7e9d9650998" (UID: "84990c2d-502d-49a9-aa52-a7e9d9650998"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:41:22.837214 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:22.837141    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84990c2d-502d-49a9-aa52-a7e9d9650998-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "84990c2d-502d-49a9-aa52-a7e9d9650998" (UID: "84990c2d-502d-49a9-aa52-a7e9d9650998"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:41:22.837214 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:22.837199    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84990c2d-502d-49a9-aa52-a7e9d9650998-service-ca" (OuterVolumeSpecName: "service-ca") pod "84990c2d-502d-49a9-aa52-a7e9d9650998" (UID: "84990c2d-502d-49a9-aa52-a7e9d9650998"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:41:22.839014 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:22.838988    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84990c2d-502d-49a9-aa52-a7e9d9650998-kube-api-access-4j2hf" (OuterVolumeSpecName: "kube-api-access-4j2hf") pod "84990c2d-502d-49a9-aa52-a7e9d9650998" (UID: "84990c2d-502d-49a9-aa52-a7e9d9650998"). InnerVolumeSpecName "kube-api-access-4j2hf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 14:41:22.839121 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:22.839002    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84990c2d-502d-49a9-aa52-a7e9d9650998-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "84990c2d-502d-49a9-aa52-a7e9d9650998" (UID: "84990c2d-502d-49a9-aa52-a7e9d9650998"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 14:41:22.839121 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:22.839055    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84990c2d-502d-49a9-aa52-a7e9d9650998-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "84990c2d-502d-49a9-aa52-a7e9d9650998" (UID: "84990c2d-502d-49a9-aa52-a7e9d9650998"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 14:41:22.937519 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:22.937478    2551 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84990c2d-502d-49a9-aa52-a7e9d9650998-console-oauth-config\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:41:22.937519 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:22.937513    2551 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84990c2d-502d-49a9-aa52-a7e9d9650998-console-config\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:41:22.937519 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:22.937522    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4j2hf\" (UniqueName: \"kubernetes.io/projected/84990c2d-502d-49a9-aa52-a7e9d9650998-kube-api-access-4j2hf\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:41:22.937519 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:22.937533    2551 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84990c2d-502d-49a9-aa52-a7e9d9650998-oauth-serving-cert\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:41:22.937813 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:22.937542    2551 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84990c2d-502d-49a9-aa52-a7e9d9650998-console-serving-cert\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:41:22.937813 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:22.937550    2551 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84990c2d-502d-49a9-aa52-a7e9d9650998-service-ca\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:41:23.671696 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:23.671653    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d64bb8994-n2p5m" event={"ID":"84990c2d-502d-49a9-aa52-a7e9d9650998","Type":"ContainerDied","Data":"880e32b8660351eb74e66febefcb62bb665cc0583fd3546cc4773f268a5fcf54"}
Jul 01 14:41:23.671696 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:23.671685    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d64bb8994-n2p5m"
Jul 01 14:41:23.672146 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:23.671727    2551 scope.go:117] "RemoveContainer" containerID="b4c17a93ece8410076ebcf8ee1968eec34343f43282791898166e38c572d9271"
Jul 01 14:41:23.688081 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:23.688047    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d64bb8994-n2p5m"]
Jul 01 14:41:23.692557 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:23.692532    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d64bb8994-n2p5m"]
Jul 01 14:41:25.103108 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:41:25.103056    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84990c2d-502d-49a9-aa52-a7e9d9650998" path="/var/lib/kubelet/pods/84990c2d-502d-49a9-aa52-a7e9d9650998/volumes"
Jul 01 14:42:09.477442 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.477411    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-54c488f99b-mbgp2"]
Jul 01 14:42:09.477897 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.477675    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4470e2b-27f5-49b2-b979-e72a8e31eddc" containerName="registry"
Jul 01 14:42:09.477897 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.477686    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4470e2b-27f5-49b2-b979-e72a8e31eddc" containerName="registry"
Jul 01 14:42:09.477897 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.477696    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c2198a5-7d68-401c-8889-6ab86dd9ac4f" containerName="console"
Jul 01 14:42:09.477897 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.477702    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c2198a5-7d68-401c-8889-6ab86dd9ac4f" containerName="console"
Jul 01 14:42:09.477897 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.477710    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84990c2d-502d-49a9-aa52-a7e9d9650998" containerName="console"
Jul 01 14:42:09.477897 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.477716    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="84990c2d-502d-49a9-aa52-a7e9d9650998" containerName="console"
Jul 01 14:42:09.477897 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.477761    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c2198a5-7d68-401c-8889-6ab86dd9ac4f" containerName="console"
Jul 01 14:42:09.477897 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.477769    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4470e2b-27f5-49b2-b979-e72a8e31eddc" containerName="registry"
Jul 01 14:42:09.477897 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.477775    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="84990c2d-502d-49a9-aa52-a7e9d9650998" containerName="console"
Jul 01 14:42:09.479806 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.479790    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54c488f99b-mbgp2"
Jul 01 14:42:09.485445 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.485413    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-x7qb4\""
Jul 01 14:42:09.485609 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.485483    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 01 14:42:09.485609 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.485534    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 01 14:42:09.485609 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.485484    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 01 14:42:09.485810 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.485791    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 01 14:42:09.486945 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.486929    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 01 14:42:09.493493 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.493472    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 01 14:42:09.504315 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.504291    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54c488f99b-mbgp2"]
Jul 01 14:42:09.600793 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.600758    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ac66ecb-8739-4826-b342-c4d6c75a2f96-oauth-serving-cert\") pod \"console-54c488f99b-mbgp2\" (UID: \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\") " pod="openshift-console/console-54c488f99b-mbgp2"
Jul 01 14:42:09.600793 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.600797    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ac66ecb-8739-4826-b342-c4d6c75a2f96-service-ca\") pod \"console-54c488f99b-mbgp2\" (UID: \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\") " pod="openshift-console/console-54c488f99b-mbgp2"
Jul 01 14:42:09.601004 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.600869    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ac66ecb-8739-4826-b342-c4d6c75a2f96-console-config\") pod \"console-54c488f99b-mbgp2\" (UID: \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\") " pod="openshift-console/console-54c488f99b-mbgp2"
Jul 01 14:42:09.601004 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.600911    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ac66ecb-8739-4826-b342-c4d6c75a2f96-trusted-ca-bundle\") pod \"console-54c488f99b-mbgp2\" (UID: \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\") " pod="openshift-console/console-54c488f99b-mbgp2"
Jul 01 14:42:09.601004 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.600937    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdxzz\" (UniqueName: \"kubernetes.io/projected/1ac66ecb-8739-4826-b342-c4d6c75a2f96-kube-api-access-rdxzz\") pod \"console-54c488f99b-mbgp2\" (UID: \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\") " pod="openshift-console/console-54c488f99b-mbgp2"
Jul 01 14:42:09.601004 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.600967    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ac66ecb-8739-4826-b342-c4d6c75a2f96-console-oauth-config\") pod \"console-54c488f99b-mbgp2\" (UID: \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\") " pod="openshift-console/console-54c488f99b-mbgp2"
Jul 01 14:42:09.601004 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.600997    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ac66ecb-8739-4826-b342-c4d6c75a2f96-console-serving-cert\") pod \"console-54c488f99b-mbgp2\" (UID: \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\") " pod="openshift-console/console-54c488f99b-mbgp2"
Jul 01 14:42:09.702074 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.702041    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ac66ecb-8739-4826-b342-c4d6c75a2f96-console-oauth-config\") pod \"console-54c488f99b-mbgp2\" (UID: \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\") " pod="openshift-console/console-54c488f99b-mbgp2"
Jul 01 14:42:09.702235 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.702085    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ac66ecb-8739-4826-b342-c4d6c75a2f96-console-serving-cert\") pod \"console-54c488f99b-mbgp2\" (UID: \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\") " pod="openshift-console/console-54c488f99b-mbgp2"
Jul 01 14:42:09.702235 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.702108    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ac66ecb-8739-4826-b342-c4d6c75a2f96-oauth-serving-cert\") pod \"console-54c488f99b-mbgp2\" (UID: \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\") " pod="openshift-console/console-54c488f99b-mbgp2"
Jul 01 14:42:09.702235 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.702126    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ac66ecb-8739-4826-b342-c4d6c75a2f96-service-ca\") pod \"console-54c488f99b-mbgp2\" (UID: \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\") " pod="openshift-console/console-54c488f99b-mbgp2"
Jul 01 14:42:09.702235 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.702157    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ac66ecb-8739-4826-b342-c4d6c75a2f96-console-config\") pod \"console-54c488f99b-mbgp2\" (UID: \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\") " pod="openshift-console/console-54c488f99b-mbgp2"
Jul 01 14:42:09.702235 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.702191    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ac66ecb-8739-4826-b342-c4d6c75a2f96-trusted-ca-bundle\") pod \"console-54c488f99b-mbgp2\" (UID: \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\") " pod="openshift-console/console-54c488f99b-mbgp2"
Jul 01 14:42:09.702235 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.702224    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdxzz\" (UniqueName: \"kubernetes.io/projected/1ac66ecb-8739-4826-b342-c4d6c75a2f96-kube-api-access-rdxzz\") pod \"console-54c488f99b-mbgp2\" (UID: \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\") " pod="openshift-console/console-54c488f99b-mbgp2"
Jul 01 14:42:09.702986 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.702960    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ac66ecb-8739-4826-b342-c4d6c75a2f96-service-ca\") pod \"console-54c488f99b-mbgp2\" (UID: \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\") " pod="openshift-console/console-54c488f99b-mbgp2"
Jul 01 14:42:09.703087 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.702989    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ac66ecb-8739-4826-b342-c4d6c75a2f96-oauth-serving-cert\") pod \"console-54c488f99b-mbgp2\" (UID: \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\") " pod="openshift-console/console-54c488f99b-mbgp2"
Jul 01 14:42:09.703087 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.702962    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ac66ecb-8739-4826-b342-c4d6c75a2f96-console-config\") pod \"console-54c488f99b-mbgp2\" (UID: \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\") " pod="openshift-console/console-54c488f99b-mbgp2"
Jul 01 14:42:09.703243 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.703222    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ac66ecb-8739-4826-b342-c4d6c75a2f96-trusted-ca-bundle\") pod \"console-54c488f99b-mbgp2\" (UID: \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\") " pod="openshift-console/console-54c488f99b-mbgp2"
Jul 01 14:42:09.704587 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.704560    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ac66ecb-8739-4826-b342-c4d6c75a2f96-console-serving-cert\") pod \"console-54c488f99b-mbgp2\" (UID: \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\") " pod="openshift-console/console-54c488f99b-mbgp2"
Jul 01 14:42:09.704688 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.704594    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ac66ecb-8739-4826-b342-c4d6c75a2f96-console-oauth-config\") pod \"console-54c488f99b-mbgp2\" (UID: \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\") " pod="openshift-console/console-54c488f99b-mbgp2"
Jul 01 14:42:09.712254 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.712233    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdxzz\" (UniqueName: \"kubernetes.io/projected/1ac66ecb-8739-4826-b342-c4d6c75a2f96-kube-api-access-rdxzz\") pod \"console-54c488f99b-mbgp2\" (UID: \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\") " pod="openshift-console/console-54c488f99b-mbgp2"
Jul 01 14:42:09.789418 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.789327    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54c488f99b-mbgp2"
Jul 01 14:42:09.928079 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:09.928044    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54c488f99b-mbgp2"]
Jul 01 14:42:09.930916 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:42:09.930891    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ac66ecb_8739_4826_b342_c4d6c75a2f96.slice/crio-904d0af4541289dc4b57b7ae784ce495fbc3f8938379014cc7b71f19bea5f80d WatchSource:0}: Error finding container 904d0af4541289dc4b57b7ae784ce495fbc3f8938379014cc7b71f19bea5f80d: Status 404 returned error can't find the container with id 904d0af4541289dc4b57b7ae784ce495fbc3f8938379014cc7b71f19bea5f80d
Jul 01 14:42:10.856462 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:10.856419    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54c488f99b-mbgp2" event={"ID":"1ac66ecb-8739-4826-b342-c4d6c75a2f96","Type":"ContainerStarted","Data":"2493f435c0e9db2503f3de2d2599bb0c8a15842fd4e2c8b234da31a42c5e2f3f"}
Jul 01 14:42:10.856877 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:10.856484    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54c488f99b-mbgp2" event={"ID":"1ac66ecb-8739-4826-b342-c4d6c75a2f96","Type":"ContainerStarted","Data":"904d0af4541289dc4b57b7ae784ce495fbc3f8938379014cc7b71f19bea5f80d"}
Jul 01 14:42:10.878336 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:10.878280    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-54c488f99b-mbgp2" podStartSLOduration=1.878262119 podStartE2EDuration="1.878262119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:42:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:42:10.87709486 +0000 UTC m=+168.205667168" watchObservedRunningTime="2026-07-01 14:42:10.878262119 +0000 UTC m=+168.206834490"
Jul 01 14:42:19.790423 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:19.790335    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-54c488f99b-mbgp2"
Jul 01 14:42:19.790423 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:19.790425    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-54c488f99b-mbgp2"
Jul 01 14:42:19.795257 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:19.795235    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-54c488f99b-mbgp2"
Jul 01 14:42:19.890060 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:42:19.890035    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-54c488f99b-mbgp2"
Jul 01 14:43:34.109231 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:43:34.109193    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54c488f99b-mbgp2"]
Jul 01 14:43:59.129703 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:43:59.129577    2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-54c488f99b-mbgp2" podUID="1ac66ecb-8739-4826-b342-c4d6c75a2f96" containerName="console" containerID="cri-o://2493f435c0e9db2503f3de2d2599bb0c8a15842fd4e2c8b234da31a42c5e2f3f" gracePeriod=15
Jul 01 14:43:59.366324 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:43:59.366295    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54c488f99b-mbgp2"
Jul 01 14:43:59.428782 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:43:59.428688    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ac66ecb-8739-4826-b342-c4d6c75a2f96-oauth-serving-cert\") pod \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\" (UID: \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\") "
Jul 01 14:43:59.428782 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:43:59.428734    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ac66ecb-8739-4826-b342-c4d6c75a2f96-service-ca\") pod \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\" (UID: \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\") "
Jul 01 14:43:59.428782 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:43:59.428762    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ac66ecb-8739-4826-b342-c4d6c75a2f96-console-config\") pod \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\" (UID: \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\") "
Jul 01 14:43:59.429029 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:43:59.428795    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ac66ecb-8739-4826-b342-c4d6c75a2f96-console-oauth-config\") pod \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\" (UID: \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\") "
Jul 01 14:43:59.429029 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:43:59.428845    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ac66ecb-8739-4826-b342-c4d6c75a2f96-console-serving-cert\") pod \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\" (UID: \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\") "
Jul 01 14:43:59.429029 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:43:59.428886    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdxzz\" (UniqueName: \"kubernetes.io/projected/1ac66ecb-8739-4826-b342-c4d6c75a2f96-kube-api-access-rdxzz\") pod \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\" (UID: \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\") "
Jul 01 14:43:59.429029 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:43:59.428923    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ac66ecb-8739-4826-b342-c4d6c75a2f96-trusted-ca-bundle\") pod \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\" (UID: \"1ac66ecb-8739-4826-b342-c4d6c75a2f96\") "
Jul 01 14:43:59.429243 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:43:59.429140    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ac66ecb-8739-4826-b342-c4d6c75a2f96-console-config" (OuterVolumeSpecName: "console-config") pod "1ac66ecb-8739-4826-b342-c4d6c75a2f96" (UID: "1ac66ecb-8739-4826-b342-c4d6c75a2f96"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:43:59.429307 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:43:59.429276    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ac66ecb-8739-4826-b342-c4d6c75a2f96-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1ac66ecb-8739-4826-b342-c4d6c75a2f96" (UID: "1ac66ecb-8739-4826-b342-c4d6c75a2f96"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:43:59.429438 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:43:59.429417    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ac66ecb-8739-4826-b342-c4d6c75a2f96-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1ac66ecb-8739-4826-b342-c4d6c75a2f96" (UID: "1ac66ecb-8739-4826-b342-c4d6c75a2f96"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:43:59.429438 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:43:59.429426    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ac66ecb-8739-4826-b342-c4d6c75a2f96-service-ca" (OuterVolumeSpecName: "service-ca") pod "1ac66ecb-8739-4826-b342-c4d6c75a2f96" (UID: "1ac66ecb-8739-4826-b342-c4d6c75a2f96"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:43:59.431143 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:43:59.431111    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ac66ecb-8739-4826-b342-c4d6c75a2f96-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1ac66ecb-8739-4826-b342-c4d6c75a2f96" (UID: "1ac66ecb-8739-4826-b342-c4d6c75a2f96"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 14:43:59.431278 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:43:59.431189    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ac66ecb-8739-4826-b342-c4d6c75a2f96-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1ac66ecb-8739-4826-b342-c4d6c75a2f96" (UID: "1ac66ecb-8739-4826-b342-c4d6c75a2f96"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 14:43:59.431340 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:43:59.431281    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ac66ecb-8739-4826-b342-c4d6c75a2f96-kube-api-access-rdxzz" (OuterVolumeSpecName: "kube-api-access-rdxzz") pod "1ac66ecb-8739-4826-b342-c4d6c75a2f96" (UID: "1ac66ecb-8739-4826-b342-c4d6c75a2f96"). InnerVolumeSpecName "kube-api-access-rdxzz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 14:43:59.530018 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:43:59.529974    2551 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ac66ecb-8739-4826-b342-c4d6c75a2f96-console-serving-cert\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:43:59.530018 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:43:59.530006    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdxzz\" (UniqueName: \"kubernetes.io/projected/1ac66ecb-8739-4826-b342-c4d6c75a2f96-kube-api-access-rdxzz\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:43:59.530018 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:43:59.530017    2551 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ac66ecb-8739-4826-b342-c4d6c75a2f96-trusted-ca-bundle\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:43:59.530018 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:43:59.530025    2551 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ac66ecb-8739-4826-b342-c4d6c75a2f96-oauth-serving-cert\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:43:59.530272 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:43:59.530035    2551 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ac66ecb-8739-4826-b342-c4d6c75a2f96-service-ca\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:43:59.530272 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:43:59.530043    2551 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ac66ecb-8739-4826-b342-c4d6c75a2f96-console-config\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:43:59.530272 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:43:59.530051    2551 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ac66ecb-8739-4826-b342-c4d6c75a2f96-console-oauth-config\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:44:00.227202 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:00.227169    2551 generic.go:356] "Generic (PLEG): container finished" podID="1ac66ecb-8739-4826-b342-c4d6c75a2f96" containerID="2493f435c0e9db2503f3de2d2599bb0c8a15842fd4e2c8b234da31a42c5e2f3f" exitCode=0
Jul 01 14:44:00.227576 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:00.227211    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54c488f99b-mbgp2"
Jul 01 14:44:00.227576 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:00.227248    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54c488f99b-mbgp2" event={"ID":"1ac66ecb-8739-4826-b342-c4d6c75a2f96","Type":"ContainerDied","Data":"2493f435c0e9db2503f3de2d2599bb0c8a15842fd4e2c8b234da31a42c5e2f3f"}
Jul 01 14:44:00.227576 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:00.227299    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54c488f99b-mbgp2" event={"ID":"1ac66ecb-8739-4826-b342-c4d6c75a2f96","Type":"ContainerDied","Data":"904d0af4541289dc4b57b7ae784ce495fbc3f8938379014cc7b71f19bea5f80d"}
Jul 01 14:44:00.227576 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:00.227321    2551 scope.go:117] "RemoveContainer" containerID="2493f435c0e9db2503f3de2d2599bb0c8a15842fd4e2c8b234da31a42c5e2f3f"
Jul 01 14:44:00.235232 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:00.235213    2551 scope.go:117] "RemoveContainer" containerID="2493f435c0e9db2503f3de2d2599bb0c8a15842fd4e2c8b234da31a42c5e2f3f"
Jul 01 14:44:00.235509 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:44:00.235485    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2493f435c0e9db2503f3de2d2599bb0c8a15842fd4e2c8b234da31a42c5e2f3f\": container with ID starting with 2493f435c0e9db2503f3de2d2599bb0c8a15842fd4e2c8b234da31a42c5e2f3f not found: ID does not exist" containerID="2493f435c0e9db2503f3de2d2599bb0c8a15842fd4e2c8b234da31a42c5e2f3f"
Jul 01 14:44:00.235559 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:00.235518    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2493f435c0e9db2503f3de2d2599bb0c8a15842fd4e2c8b234da31a42c5e2f3f"} err="failed to get container status \"2493f435c0e9db2503f3de2d2599bb0c8a15842fd4e2c8b234da31a42c5e2f3f\": rpc error: code = NotFound desc = could not find container \"2493f435c0e9db2503f3de2d2599bb0c8a15842fd4e2c8b234da31a42c5e2f3f\": container with ID starting with 2493f435c0e9db2503f3de2d2599bb0c8a15842fd4e2c8b234da31a42c5e2f3f not found: ID does not exist"
Jul 01 14:44:00.247452 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:00.247426    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54c488f99b-mbgp2"]
Jul 01 14:44:00.248928 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:00.248905    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-54c488f99b-mbgp2"]
Jul 01 14:44:01.101953 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:01.101921    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ac66ecb-8739-4826-b342-c4d6c75a2f96" path="/var/lib/kubelet/pods/1ac66ecb-8739-4826-b342-c4d6c75a2f96/volumes"
Jul 01 14:44:23.038751 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:23.038720    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/ovn-acl-logging/0.log"
Jul 01 14:44:23.043718 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:23.043687    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/ovn-acl-logging/0.log"
Jul 01 14:44:23.044340 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:23.044316    2551 kubelet.go:1648] "Image garbage collection succeeded"
Jul 01 14:44:25.666642 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:25.666593    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589b27f5"]
Jul 01 14:44:25.668265 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:25.667010    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ac66ecb-8739-4826-b342-c4d6c75a2f96" containerName="console"
Jul 01 14:44:25.668265 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:25.667028    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ac66ecb-8739-4826-b342-c4d6c75a2f96" containerName="console"
Jul 01 14:44:25.668265 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:25.667097    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ac66ecb-8739-4826-b342-c4d6c75a2f96" containerName="console"
Jul 01 14:44:25.668916 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:25.668899    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589b27f5"
Jul 01 14:44:25.671424 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:25.671399    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 01 14:44:25.671562 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:25.671430    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 01 14:44:25.671562 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:25.671504    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zhcjp\""
Jul 01 14:44:25.681080 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:25.681054    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589b27f5"]
Jul 01 14:44:25.810161 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:25.810118    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed91871d-f290-4b36-9cc3-2927d9cbd304-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589b27f5\" (UID: \"ed91871d-f290-4b36-9cc3-2927d9cbd304\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589b27f5"
Jul 01 14:44:25.810161 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:25.810159    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed91871d-f290-4b36-9cc3-2927d9cbd304-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589b27f5\" (UID: \"ed91871d-f290-4b36-9cc3-2927d9cbd304\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589b27f5"
Jul 01 14:44:25.810369 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:25.810264    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxnz2\" (UniqueName: \"kubernetes.io/projected/ed91871d-f290-4b36-9cc3-2927d9cbd304-kube-api-access-rxnz2\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589b27f5\" (UID: \"ed91871d-f290-4b36-9cc3-2927d9cbd304\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589b27f5"
Jul 01 14:44:25.911507 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:25.911463    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxnz2\" (UniqueName: \"kubernetes.io/projected/ed91871d-f290-4b36-9cc3-2927d9cbd304-kube-api-access-rxnz2\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589b27f5\" (UID: \"ed91871d-f290-4b36-9cc3-2927d9cbd304\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589b27f5"
Jul 01 14:44:25.911686 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:25.911515    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed91871d-f290-4b36-9cc3-2927d9cbd304-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589b27f5\" (UID: \"ed91871d-f290-4b36-9cc3-2927d9cbd304\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589b27f5"
Jul 01 14:44:25.911686 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:25.911534    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed91871d-f290-4b36-9cc3-2927d9cbd304-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589b27f5\" (UID: \"ed91871d-f290-4b36-9cc3-2927d9cbd304\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589b27f5"
Jul 01 14:44:25.911912 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:25.911892    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed91871d-f290-4b36-9cc3-2927d9cbd304-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589b27f5\" (UID: \"ed91871d-f290-4b36-9cc3-2927d9cbd304\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589b27f5"
Jul 01 14:44:25.911955 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:25.911913    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed91871d-f290-4b36-9cc3-2927d9cbd304-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589b27f5\" (UID: \"ed91871d-f290-4b36-9cc3-2927d9cbd304\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589b27f5"
Jul 01 14:44:25.920467 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:25.920401    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxnz2\" (UniqueName: \"kubernetes.io/projected/ed91871d-f290-4b36-9cc3-2927d9cbd304-kube-api-access-rxnz2\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589b27f5\" (UID: \"ed91871d-f290-4b36-9cc3-2927d9cbd304\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589b27f5"
Jul 01 14:44:25.978586 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:25.978553    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589b27f5"
Jul 01 14:44:26.110806 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:26.110704    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589b27f5"]
Jul 01 14:44:26.113359 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:44:26.113331    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded91871d_f290_4b36_9cc3_2927d9cbd304.slice/crio-c360b5afcc9959141b1ff0b5f9739f9c73e5fa438bc1dc9872dc803c49a0525f WatchSource:0}: Error finding container c360b5afcc9959141b1ff0b5f9739f9c73e5fa438bc1dc9872dc803c49a0525f: Status 404 returned error can't find the container with id c360b5afcc9959141b1ff0b5f9739f9c73e5fa438bc1dc9872dc803c49a0525f
Jul 01 14:44:26.115340 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:26.115324    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 01 14:44:26.317936 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:26.317903    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589b27f5" event={"ID":"ed91871d-f290-4b36-9cc3-2927d9cbd304","Type":"ContainerStarted","Data":"c360b5afcc9959141b1ff0b5f9739f9c73e5fa438bc1dc9872dc803c49a0525f"}
Jul 01 14:44:34.349338 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:34.349243    2551 generic.go:356] "Generic (PLEG): container finished" podID="ed91871d-f290-4b36-9cc3-2927d9cbd304" containerID="06f304af3ddd095ca57ddac92bee71c9edc4d44267da349f3317b33f3a69290d" exitCode=0
Jul 01 14:44:34.349338 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:34.349327    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589b27f5" event={"ID":"ed91871d-f290-4b36-9cc3-2927d9cbd304","Type":"ContainerDied","Data":"06f304af3ddd095ca57ddac92bee71c9edc4d44267da349f3317b33f3a69290d"}
Jul 01 14:44:37.366541 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:37.366488    2551 generic.go:356] "Generic (PLEG): container finished" podID="ed91871d-f290-4b36-9cc3-2927d9cbd304" containerID="48462fdbf5d271ff4cbfdf15238b42d7bd993d0f493f6856cc9894f4fa5824ec" exitCode=0
Jul 01 14:44:37.366999 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:37.366665    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589b27f5" event={"ID":"ed91871d-f290-4b36-9cc3-2927d9cbd304","Type":"ContainerDied","Data":"48462fdbf5d271ff4cbfdf15238b42d7bd993d0f493f6856cc9894f4fa5824ec"}
Jul 01 14:44:45.398570 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:45.398459    2551 generic.go:356] "Generic (PLEG): container finished" podID="ed91871d-f290-4b36-9cc3-2927d9cbd304" containerID="9229e09746fa8d12f290dde71555309fe2c7871787a0c85513269ac022d83963" exitCode=0
Jul 01 14:44:45.398570 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:45.398537    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589b27f5" event={"ID":"ed91871d-f290-4b36-9cc3-2927d9cbd304","Type":"ContainerDied","Data":"9229e09746fa8d12f290dde71555309fe2c7871787a0c85513269ac022d83963"}
Jul 01 14:44:46.574745 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:46.574717    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589b27f5"
Jul 01 14:44:46.673894 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:46.673789    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed91871d-f290-4b36-9cc3-2927d9cbd304-util\") pod \"ed91871d-f290-4b36-9cc3-2927d9cbd304\" (UID: \"ed91871d-f290-4b36-9cc3-2927d9cbd304\") "
Jul 01 14:44:46.673894 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:46.673835    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxnz2\" (UniqueName: \"kubernetes.io/projected/ed91871d-f290-4b36-9cc3-2927d9cbd304-kube-api-access-rxnz2\") pod \"ed91871d-f290-4b36-9cc3-2927d9cbd304\" (UID: \"ed91871d-f290-4b36-9cc3-2927d9cbd304\") "
Jul 01 14:44:46.673894 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:46.673860    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed91871d-f290-4b36-9cc3-2927d9cbd304-bundle\") pod \"ed91871d-f290-4b36-9cc3-2927d9cbd304\" (UID: \"ed91871d-f290-4b36-9cc3-2927d9cbd304\") "
Jul 01 14:44:46.674684 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:46.674649    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed91871d-f290-4b36-9cc3-2927d9cbd304-bundle" (OuterVolumeSpecName: "bundle") pod "ed91871d-f290-4b36-9cc3-2927d9cbd304" (UID: "ed91871d-f290-4b36-9cc3-2927d9cbd304"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 14:44:46.676144 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:46.676118    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed91871d-f290-4b36-9cc3-2927d9cbd304-kube-api-access-rxnz2" (OuterVolumeSpecName: "kube-api-access-rxnz2") pod "ed91871d-f290-4b36-9cc3-2927d9cbd304" (UID: "ed91871d-f290-4b36-9cc3-2927d9cbd304"). InnerVolumeSpecName "kube-api-access-rxnz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 14:44:46.678067 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:46.678042    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed91871d-f290-4b36-9cc3-2927d9cbd304-util" (OuterVolumeSpecName: "util") pod "ed91871d-f290-4b36-9cc3-2927d9cbd304" (UID: "ed91871d-f290-4b36-9cc3-2927d9cbd304"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 14:44:46.774551 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:46.774511    2551 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed91871d-f290-4b36-9cc3-2927d9cbd304-util\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:44:46.774551 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:46.774541    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxnz2\" (UniqueName: \"kubernetes.io/projected/ed91871d-f290-4b36-9cc3-2927d9cbd304-kube-api-access-rxnz2\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:44:46.774551 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:46.774552    2551 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed91871d-f290-4b36-9cc3-2927d9cbd304-bundle\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:44:47.406858 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:47.406772    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589b27f5" event={"ID":"ed91871d-f290-4b36-9cc3-2927d9cbd304","Type":"ContainerDied","Data":"c360b5afcc9959141b1ff0b5f9739f9c73e5fa438bc1dc9872dc803c49a0525f"}
Jul 01 14:44:47.406858 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:47.406822    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c360b5afcc9959141b1ff0b5f9739f9c73e5fa438bc1dc9872dc803c49a0525f"
Jul 01 14:44:47.406858 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:47.406780    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589b27f5"
Jul 01 14:44:53.738670 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:53.738635    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-sv7hk"]
Jul 01 14:44:53.739072 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:53.738881    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed91871d-f290-4b36-9cc3-2927d9cbd304" containerName="extract"
Jul 01 14:44:53.739072 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:53.738891    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed91871d-f290-4b36-9cc3-2927d9cbd304" containerName="extract"
Jul 01 14:44:53.739072 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:53.738902    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed91871d-f290-4b36-9cc3-2927d9cbd304" containerName="util"
Jul 01 14:44:53.739072 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:53.738908    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed91871d-f290-4b36-9cc3-2927d9cbd304" containerName="util"
Jul 01 14:44:53.739072 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:53.738917    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed91871d-f290-4b36-9cc3-2927d9cbd304" containerName="pull"
Jul 01 14:44:53.739072 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:53.738922    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed91871d-f290-4b36-9cc3-2927d9cbd304" containerName="pull"
Jul 01 14:44:53.739072 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:53.738959    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed91871d-f290-4b36-9cc3-2927d9cbd304" containerName="extract"
Jul 01 14:44:53.740792 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:53.740770    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-sv7hk"
Jul 01 14:44:53.743468 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:53.743450    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Jul 01 14:44:53.743582 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:53.743529    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-t2b8m\""
Jul 01 14:44:53.743582 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:53.743550    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Jul 01 14:44:53.743582 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:53.743570    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Jul 01 14:44:53.753316 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:53.753292    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-sv7hk"]
Jul 01 14:44:53.816594 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:53.816558    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn9v8\" (UniqueName: \"kubernetes.io/projected/24da3a00-546d-4c59-af97-a05cc6ed8d51-kube-api-access-fn9v8\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-sv7hk\" (UID: \"24da3a00-546d-4c59-af97-a05cc6ed8d51\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-sv7hk"
Jul 01 14:44:53.816797 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:53.816602    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/24da3a00-546d-4c59-af97-a05cc6ed8d51-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-sv7hk\" (UID: \"24da3a00-546d-4c59-af97-a05cc6ed8d51\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-sv7hk"
Jul 01 14:44:53.916973 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:53.916948    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn9v8\" (UniqueName: \"kubernetes.io/projected/24da3a00-546d-4c59-af97-a05cc6ed8d51-kube-api-access-fn9v8\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-sv7hk\" (UID: \"24da3a00-546d-4c59-af97-a05cc6ed8d51\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-sv7hk"
Jul 01 14:44:53.917078 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:53.916983    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/24da3a00-546d-4c59-af97-a05cc6ed8d51-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-sv7hk\" (UID: \"24da3a00-546d-4c59-af97-a05cc6ed8d51\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-sv7hk"
Jul 01 14:44:53.919283 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:53.919258    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/24da3a00-546d-4c59-af97-a05cc6ed8d51-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-sv7hk\" (UID: \"24da3a00-546d-4c59-af97-a05cc6ed8d51\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-sv7hk"
Jul 01 14:44:53.926437 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:53.926413    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn9v8\" (UniqueName: \"kubernetes.io/projected/24da3a00-546d-4c59-af97-a05cc6ed8d51-kube-api-access-fn9v8\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-sv7hk\" (UID: \"24da3a00-546d-4c59-af97-a05cc6ed8d51\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-sv7hk"
Jul 01 14:44:54.050216 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:54.050129    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-sv7hk"
Jul 01 14:44:54.186155 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:54.186072    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-sv7hk"]
Jul 01 14:44:54.191136 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:44:54.191095    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24da3a00_546d_4c59_af97_a05cc6ed8d51.slice/crio-a7a049d774aa6f787153f4652118385ad1b3a36900bb5f895fc6b25af6b55597 WatchSource:0}: Error finding container a7a049d774aa6f787153f4652118385ad1b3a36900bb5f895fc6b25af6b55597: Status 404 returned error can't find the container with id a7a049d774aa6f787153f4652118385ad1b3a36900bb5f895fc6b25af6b55597
Jul 01 14:44:54.432039 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:54.431958    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-sv7hk" event={"ID":"24da3a00-546d-4c59-af97-a05cc6ed8d51","Type":"ContainerStarted","Data":"a7a049d774aa6f787153f4652118385ad1b3a36900bb5f895fc6b25af6b55597"}
Jul 01 14:44:57.450202 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:57.450100    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-sv7hk" event={"ID":"24da3a00-546d-4c59-af97-a05cc6ed8d51","Type":"ContainerStarted","Data":"b1dada471ec359bf1a9434f5295d2e2d08dcd81d3b122bf4f72e74e3a637a94a"}
Jul 01 14:44:57.450666 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:57.450250    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-sv7hk"
Jul 01 14:44:57.474058 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:57.474004    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-sv7hk" podStartSLOduration=1.662633013 podStartE2EDuration="4.473987367s" totalImagesPullingTime="2.811354354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:44:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:44:57.471668483 +0000 UTC m=+334.800240783" watchObservedRunningTime="2026-07-01 14:44:57.473987367 +0000 UTC m=+334.802559672"
Jul 01 14:44:57.821478 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:57.821444    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-68bff59c-gj7sr"]
Jul 01 14:44:57.823775 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:57.823751    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-68bff59c-gj7sr"
Jul 01 14:44:57.826238 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:57.826214    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Jul 01 14:44:57.826238 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:57.826224    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Jul 01 14:44:57.826415 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:57.826214    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-m5k4n\""
Jul 01 14:44:57.834500 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:57.834470    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-68bff59c-gj7sr"]
Jul 01 14:44:57.844504 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:57.844470    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d9909e9b-af80-4cfa-90cb-28cb9c9cdce1-certificates\") pod \"keda-operator-68bff59c-gj7sr\" (UID: \"d9909e9b-af80-4cfa-90cb-28cb9c9cdce1\") " pod="openshift-keda/keda-operator-68bff59c-gj7sr"
Jul 01 14:44:57.844504 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:57.844508    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpdq7\" (UniqueName: \"kubernetes.io/projected/d9909e9b-af80-4cfa-90cb-28cb9c9cdce1-kube-api-access-dpdq7\") pod \"keda-operator-68bff59c-gj7sr\" (UID: \"d9909e9b-af80-4cfa-90cb-28cb9c9cdce1\") " pod="openshift-keda/keda-operator-68bff59c-gj7sr"
Jul 01 14:44:57.844723 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:57.844529    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/d9909e9b-af80-4cfa-90cb-28cb9c9cdce1-cabundle0\") pod \"keda-operator-68bff59c-gj7sr\" (UID: \"d9909e9b-af80-4cfa-90cb-28cb9c9cdce1\") " pod="openshift-keda/keda-operator-68bff59c-gj7sr"
Jul 01 14:44:57.945433 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:57.945392    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d9909e9b-af80-4cfa-90cb-28cb9c9cdce1-certificates\") pod \"keda-operator-68bff59c-gj7sr\" (UID: \"d9909e9b-af80-4cfa-90cb-28cb9c9cdce1\") " pod="openshift-keda/keda-operator-68bff59c-gj7sr"
Jul 01 14:44:57.945433 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:57.945431    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpdq7\" (UniqueName: \"kubernetes.io/projected/d9909e9b-af80-4cfa-90cb-28cb9c9cdce1-kube-api-access-dpdq7\") pod \"keda-operator-68bff59c-gj7sr\" (UID: \"d9909e9b-af80-4cfa-90cb-28cb9c9cdce1\") " pod="openshift-keda/keda-operator-68bff59c-gj7sr"
Jul 01 14:44:57.945713 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:57.945448    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/d9909e9b-af80-4cfa-90cb-28cb9c9cdce1-cabundle0\") pod \"keda-operator-68bff59c-gj7sr\" (UID: \"d9909e9b-af80-4cfa-90cb-28cb9c9cdce1\") " pod="openshift-keda/keda-operator-68bff59c-gj7sr"
Jul 01 14:44:57.945713 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:44:57.945554    2551 secret.go:281] references non-existent secret key: ca.crt
Jul 01 14:44:57.945713 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:44:57.945576    2551 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jul 01 14:44:57.945713 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:44:57.945587    2551 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-68bff59c-gj7sr: references non-existent secret key: ca.crt
Jul 01 14:44:57.945713 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:44:57.945673    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9909e9b-af80-4cfa-90cb-28cb9c9cdce1-certificates podName:d9909e9b-af80-4cfa-90cb-28cb9c9cdce1 nodeName:}" failed. No retries permitted until 2026-07-01 14:44:58.445652331 +0000 UTC m=+335.774224625 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/d9909e9b-af80-4cfa-90cb-28cb9c9cdce1-certificates") pod "keda-operator-68bff59c-gj7sr" (UID: "d9909e9b-af80-4cfa-90cb-28cb9c9cdce1") : references non-existent secret key: ca.crt
Jul 01 14:44:57.946073 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:57.946055    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/d9909e9b-af80-4cfa-90cb-28cb9c9cdce1-cabundle0\") pod \"keda-operator-68bff59c-gj7sr\" (UID: \"d9909e9b-af80-4cfa-90cb-28cb9c9cdce1\") " pod="openshift-keda/keda-operator-68bff59c-gj7sr"
Jul 01 14:44:57.956372 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:57.956341    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpdq7\" (UniqueName: \"kubernetes.io/projected/d9909e9b-af80-4cfa-90cb-28cb9c9cdce1-kube-api-access-dpdq7\") pod \"keda-operator-68bff59c-gj7sr\" (UID: \"d9909e9b-af80-4cfa-90cb-28cb9c9cdce1\") " pod="openshift-keda/keda-operator-68bff59c-gj7sr"
Jul 01 14:44:58.164941 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:58.164847    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-cvz26"]
Jul 01 14:44:58.167058 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:58.167042    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-cvz26"
Jul 01 14:44:58.169425 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:58.169404    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Jul 01 14:44:58.177528 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:58.177499    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-cvz26"]
Jul 01 14:44:58.247383 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:58.247342    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/e4eb0333-51d8-4ae5-959e-16049497669e-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-cvz26\" (UID: \"e4eb0333-51d8-4ae5-959e-16049497669e\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-cvz26"
Jul 01 14:44:58.247560 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:58.247402    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e4eb0333-51d8-4ae5-959e-16049497669e-certificates\") pod \"keda-metrics-apiserver-559f75f947-cvz26\" (UID: \"e4eb0333-51d8-4ae5-959e-16049497669e\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-cvz26"
Jul 01 14:44:58.247560 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:58.247452    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sxsx\" (UniqueName: \"kubernetes.io/projected/e4eb0333-51d8-4ae5-959e-16049497669e-kube-api-access-6sxsx\") pod \"keda-metrics-apiserver-559f75f947-cvz26\" (UID: \"e4eb0333-51d8-4ae5-959e-16049497669e\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-cvz26"
Jul 01 14:44:58.347800 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:58.347761    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sxsx\" (UniqueName: \"kubernetes.io/projected/e4eb0333-51d8-4ae5-959e-16049497669e-kube-api-access-6sxsx\") pod \"keda-metrics-apiserver-559f75f947-cvz26\" (UID: \"e4eb0333-51d8-4ae5-959e-16049497669e\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-cvz26"
Jul 01 14:44:58.347983 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:58.347825    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/e4eb0333-51d8-4ae5-959e-16049497669e-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-cvz26\" (UID: \"e4eb0333-51d8-4ae5-959e-16049497669e\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-cvz26"
Jul 01 14:44:58.347983 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:58.347855    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e4eb0333-51d8-4ae5-959e-16049497669e-certificates\") pod \"keda-metrics-apiserver-559f75f947-cvz26\" (UID: \"e4eb0333-51d8-4ae5-959e-16049497669e\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-cvz26"
Jul 01 14:44:58.347983 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:44:58.347963    2551 secret.go:281] references non-existent secret key: tls.crt
Jul 01 14:44:58.347983 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:44:58.347976    2551 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Jul 01 14:44:58.348202 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:44:58.347993    2551 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-559f75f947-cvz26: references non-existent secret key: tls.crt
Jul 01 14:44:58.348202 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:44:58.348042    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4eb0333-51d8-4ae5-959e-16049497669e-certificates podName:e4eb0333-51d8-4ae5-959e-16049497669e nodeName:}" failed. No retries permitted until 2026-07-01 14:44:58.848027102 +0000 UTC m=+336.176599386 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/e4eb0333-51d8-4ae5-959e-16049497669e-certificates") pod "keda-metrics-apiserver-559f75f947-cvz26" (UID: "e4eb0333-51d8-4ae5-959e-16049497669e") : references non-existent secret key: tls.crt
Jul 01 14:44:58.348318 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:58.348295    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/e4eb0333-51d8-4ae5-959e-16049497669e-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-cvz26\" (UID: \"e4eb0333-51d8-4ae5-959e-16049497669e\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-cvz26"
Jul 01 14:44:58.357197 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:58.357159    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sxsx\" (UniqueName: \"kubernetes.io/projected/e4eb0333-51d8-4ae5-959e-16049497669e-kube-api-access-6sxsx\") pod \"keda-metrics-apiserver-559f75f947-cvz26\" (UID: \"e4eb0333-51d8-4ae5-959e-16049497669e\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-cvz26"
Jul 01 14:44:58.448540 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:58.448447    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d9909e9b-af80-4cfa-90cb-28cb9c9cdce1-certificates\") pod \"keda-operator-68bff59c-gj7sr\" (UID: \"d9909e9b-af80-4cfa-90cb-28cb9c9cdce1\") " pod="openshift-keda/keda-operator-68bff59c-gj7sr"
Jul 01 14:44:58.448731 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:44:58.448600    2551 secret.go:281] references non-existent secret key: ca.crt
Jul 01 14:44:58.448731 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:44:58.448642    2551 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jul 01 14:44:58.448731 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:44:58.448652    2551 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-68bff59c-gj7sr: references non-existent secret key: ca.crt
Jul 01 14:44:58.448731 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:44:58.448706    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9909e9b-af80-4cfa-90cb-28cb9c9cdce1-certificates podName:d9909e9b-af80-4cfa-90cb-28cb9c9cdce1 nodeName:}" failed. No retries permitted until 2026-07-01 14:44:59.448691205 +0000 UTC m=+336.777263493 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/d9909e9b-af80-4cfa-90cb-28cb9c9cdce1-certificates") pod "keda-operator-68bff59c-gj7sr" (UID: "d9909e9b-af80-4cfa-90cb-28cb9c9cdce1") : references non-existent secret key: ca.crt
Jul 01 14:44:58.851826 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:58.851784    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e4eb0333-51d8-4ae5-959e-16049497669e-certificates\") pod \"keda-metrics-apiserver-559f75f947-cvz26\" (UID: \"e4eb0333-51d8-4ae5-959e-16049497669e\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-cvz26"
Jul 01 14:44:58.854349 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:58.854327    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e4eb0333-51d8-4ae5-959e-16049497669e-certificates\") pod \"keda-metrics-apiserver-559f75f947-cvz26\" (UID: \"e4eb0333-51d8-4ae5-959e-16049497669e\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-cvz26"
Jul 01 14:44:59.078404 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:59.078368    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-cvz26"
Jul 01 14:44:59.230847 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:59.230815    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-cvz26"]
Jul 01 14:44:59.234438 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:44:59.234399    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4eb0333_51d8_4ae5_959e_16049497669e.slice/crio-aa986eb88c765a37623e1697f8c6dffac2301493d4f0b0cc213f6fa0e8fa8d3a WatchSource:0}: Error finding container aa986eb88c765a37623e1697f8c6dffac2301493d4f0b0cc213f6fa0e8fa8d3a: Status 404 returned error can't find the container with id aa986eb88c765a37623e1697f8c6dffac2301493d4f0b0cc213f6fa0e8fa8d3a
Jul 01 14:44:59.460247 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:59.460146    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d9909e9b-af80-4cfa-90cb-28cb9c9cdce1-certificates\") pod \"keda-operator-68bff59c-gj7sr\" (UID: \"d9909e9b-af80-4cfa-90cb-28cb9c9cdce1\") " pod="openshift-keda/keda-operator-68bff59c-gj7sr"
Jul 01 14:44:59.463103 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:59.463068    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-cvz26" event={"ID":"e4eb0333-51d8-4ae5-959e-16049497669e","Type":"ContainerStarted","Data":"aa986eb88c765a37623e1697f8c6dffac2301493d4f0b0cc213f6fa0e8fa8d3a"}
Jul 01 14:44:59.463240 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:59.463108    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d9909e9b-af80-4cfa-90cb-28cb9c9cdce1-certificates\") pod \"keda-operator-68bff59c-gj7sr\" (UID: \"d9909e9b-af80-4cfa-90cb-28cb9c9cdce1\") " pod="openshift-keda/keda-operator-68bff59c-gj7sr"
Jul 01 14:44:59.634384 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:59.634334    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-68bff59c-gj7sr"
Jul 01 14:44:59.764149 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:44:59.764117    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-68bff59c-gj7sr"]
Jul 01 14:44:59.767046 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:44:59.767012    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9909e9b_af80_4cfa_90cb_28cb9c9cdce1.slice/crio-caccb43f317905d1502670a1f25feba24b97bf7f9c00951960ca77c4af9d8f57 WatchSource:0}: Error finding container caccb43f317905d1502670a1f25feba24b97bf7f9c00951960ca77c4af9d8f57: Status 404 returned error can't find the container with id caccb43f317905d1502670a1f25feba24b97bf7f9c00951960ca77c4af9d8f57
Jul 01 14:45:00.467835 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:45:00.467799    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-68bff59c-gj7sr" event={"ID":"d9909e9b-af80-4cfa-90cb-28cb9c9cdce1","Type":"ContainerStarted","Data":"caccb43f317905d1502670a1f25feba24b97bf7f9c00951960ca77c4af9d8f57"}
Jul 01 14:45:04.486470 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:45:04.486437    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-cvz26" event={"ID":"e4eb0333-51d8-4ae5-959e-16049497669e","Type":"ContainerStarted","Data":"cbe637bfd867ecbf66505f4f57cf4ecbce6d9de28cd947015c213b7f1dd15c14"}
Jul 01 14:45:04.486944 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:45:04.486540    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-cvz26"
Jul 01 14:45:04.487873 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:45:04.487845    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-68bff59c-gj7sr" event={"ID":"d9909e9b-af80-4cfa-90cb-28cb9c9cdce1","Type":"ContainerStarted","Data":"5bffafe582acb43aca01b0b11a576376d8108077865f8962ef18e4ec585a3705"}
Jul 01 14:45:04.487930 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:45:04.487900    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-68bff59c-gj7sr"
Jul 01 14:45:04.504808 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:45:04.504759    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-559f75f947-cvz26" podStartSLOduration=1.975913991 podStartE2EDuration="6.50474341s" totalImagesPullingTime="4.528829419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:45:04.503480534 +0000 UTC m=+341.832052845" watchObservedRunningTime="2026-07-01 14:45:04.50474341 +0000 UTC m=+341.833315713"
Jul 01 14:45:04.519792 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:45:04.519743    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-68bff59c-gj7sr" podStartSLOduration=3.518896919 podStartE2EDuration="7.519725563s" totalImagesPullingTime="4.000828644s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:44:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:45:04.518804899 +0000 UTC m=+341.847377330" watchObservedRunningTime="2026-07-01 14:45:04.519725563 +0000 UTC m=+341.848297868"
Jul 01 14:45:15.495867 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:45:15.495783    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-cvz26"
Jul 01 14:45:18.456013 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:45:18.455981    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-sv7hk"
Jul 01 14:45:25.493786 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:45:25.493760    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-68bff59c-gj7sr"
Jul 01 14:46:06.990841 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:06.990804    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-5d77c4c564-kdn42"]
Jul 01 14:46:06.993537 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:06.993510    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-5d77c4c564-kdn42"
Jul 01 14:46:06.996846 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:06.996824    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Jul 01 14:46:06.997187 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:06.997169    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-q76p7\""
Jul 01 14:46:06.997785 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:06.997770    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 01 14:46:06.997842 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:06.997786    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 01 14:46:07.020725 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:07.020688    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-5d77c4c564-kdn42"]
Jul 01 14:46:07.065263 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:07.065228    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-64568bcd49-vm2tg"]
Jul 01 14:46:07.067356 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:07.067340    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-vm2tg"
Jul 01 14:46:07.070478 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:07.070457    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Jul 01 14:46:07.070920 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:07.070904    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-kldlb\""
Jul 01 14:46:07.078993 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:07.078967    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-vm2tg"]
Jul 01 14:46:07.182944 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:07.182911    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/aa05ded8-91de-42fb-a8e4-62fc84f7498e-data\") pod \"seaweedfs-64568bcd49-vm2tg\" (UID: \"aa05ded8-91de-42fb-a8e4-62fc84f7498e\") " pod="kserve/seaweedfs-64568bcd49-vm2tg"
Jul 01 14:46:07.183153 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:07.182958    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7p48\" (UniqueName: \"kubernetes.io/projected/aa05ded8-91de-42fb-a8e4-62fc84f7498e-kube-api-access-z7p48\") pod \"seaweedfs-64568bcd49-vm2tg\" (UID: \"aa05ded8-91de-42fb-a8e4-62fc84f7498e\") " pod="kserve/seaweedfs-64568bcd49-vm2tg"
Jul 01 14:46:07.183153 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:07.182991    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tllv7\" (UniqueName: \"kubernetes.io/projected/2451d25a-2e7e-46c3-ae65-cb5a8073f23b-kube-api-access-tllv7\") pod \"kserve-controller-manager-5d77c4c564-kdn42\" (UID: \"2451d25a-2e7e-46c3-ae65-cb5a8073f23b\") " pod="kserve/kserve-controller-manager-5d77c4c564-kdn42"
Jul 01 14:46:07.183153 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:07.183074    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2451d25a-2e7e-46c3-ae65-cb5a8073f23b-cert\") pod \"kserve-controller-manager-5d77c4c564-kdn42\" (UID: \"2451d25a-2e7e-46c3-ae65-cb5a8073f23b\") " pod="kserve/kserve-controller-manager-5d77c4c564-kdn42"
Jul 01 14:46:07.284378 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:07.284276    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/aa05ded8-91de-42fb-a8e4-62fc84f7498e-data\") pod \"seaweedfs-64568bcd49-vm2tg\" (UID: \"aa05ded8-91de-42fb-a8e4-62fc84f7498e\") " pod="kserve/seaweedfs-64568bcd49-vm2tg"
Jul 01 14:46:07.284378 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:07.284338    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7p48\" (UniqueName: \"kubernetes.io/projected/aa05ded8-91de-42fb-a8e4-62fc84f7498e-kube-api-access-z7p48\") pod \"seaweedfs-64568bcd49-vm2tg\" (UID: \"aa05ded8-91de-42fb-a8e4-62fc84f7498e\") " pod="kserve/seaweedfs-64568bcd49-vm2tg"
Jul 01 14:46:07.284612 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:07.284428    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tllv7\" (UniqueName: \"kubernetes.io/projected/2451d25a-2e7e-46c3-ae65-cb5a8073f23b-kube-api-access-tllv7\") pod \"kserve-controller-manager-5d77c4c564-kdn42\" (UID: \"2451d25a-2e7e-46c3-ae65-cb5a8073f23b\") " pod="kserve/kserve-controller-manager-5d77c4c564-kdn42"
Jul 01 14:46:07.284612 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:07.284461    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2451d25a-2e7e-46c3-ae65-cb5a8073f23b-cert\") pod \"kserve-controller-manager-5d77c4c564-kdn42\" (UID: \"2451d25a-2e7e-46c3-ae65-cb5a8073f23b\") " pod="kserve/kserve-controller-manager-5d77c4c564-kdn42"
Jul 01 14:46:07.284612 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:46:07.284555    2551 secret.go:189] Couldn't get secret kserve/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 01 14:46:07.284612 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:46:07.284614    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2451d25a-2e7e-46c3-ae65-cb5a8073f23b-cert podName:2451d25a-2e7e-46c3-ae65-cb5a8073f23b nodeName:}" failed. No retries permitted until 2026-07-01 14:46:07.784598755 +0000 UTC m=+405.113171039 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2451d25a-2e7e-46c3-ae65-cb5a8073f23b-cert") pod "kserve-controller-manager-5d77c4c564-kdn42" (UID: "2451d25a-2e7e-46c3-ae65-cb5a8073f23b") : secret "kserve-webhook-server-cert" not found
Jul 01 14:46:07.284866 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:07.284786    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/aa05ded8-91de-42fb-a8e4-62fc84f7498e-data\") pod \"seaweedfs-64568bcd49-vm2tg\" (UID: \"aa05ded8-91de-42fb-a8e4-62fc84f7498e\") " pod="kserve/seaweedfs-64568bcd49-vm2tg"
Jul 01 14:46:07.298232 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:07.298201    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7p48\" (UniqueName: \"kubernetes.io/projected/aa05ded8-91de-42fb-a8e4-62fc84f7498e-kube-api-access-z7p48\") pod \"seaweedfs-64568bcd49-vm2tg\" (UID: \"aa05ded8-91de-42fb-a8e4-62fc84f7498e\") " pod="kserve/seaweedfs-64568bcd49-vm2tg"
Jul 01 14:46:07.299408 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:07.299382    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tllv7\" (UniqueName: \"kubernetes.io/projected/2451d25a-2e7e-46c3-ae65-cb5a8073f23b-kube-api-access-tllv7\") pod \"kserve-controller-manager-5d77c4c564-kdn42\" (UID: \"2451d25a-2e7e-46c3-ae65-cb5a8073f23b\") " pod="kserve/kserve-controller-manager-5d77c4c564-kdn42"
Jul 01 14:46:07.376754 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:07.376707    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-vm2tg"
Jul 01 14:46:07.527357 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:07.527313    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-vm2tg"]
Jul 01 14:46:07.529563 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:46:07.529536    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa05ded8_91de_42fb_a8e4_62fc84f7498e.slice/crio-89f13c982ae1d2f14b10d28b6247c3a8551a46cbc030764e521832414b8da56e WatchSource:0}: Error finding container 89f13c982ae1d2f14b10d28b6247c3a8551a46cbc030764e521832414b8da56e: Status 404 returned error can't find the container with id 89f13c982ae1d2f14b10d28b6247c3a8551a46cbc030764e521832414b8da56e
Jul 01 14:46:07.718833 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:07.718737    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-vm2tg" event={"ID":"aa05ded8-91de-42fb-a8e4-62fc84f7498e","Type":"ContainerStarted","Data":"89f13c982ae1d2f14b10d28b6247c3a8551a46cbc030764e521832414b8da56e"}
Jul 01 14:46:07.789367 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:07.789331    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2451d25a-2e7e-46c3-ae65-cb5a8073f23b-cert\") pod \"kserve-controller-manager-5d77c4c564-kdn42\" (UID: \"2451d25a-2e7e-46c3-ae65-cb5a8073f23b\") " pod="kserve/kserve-controller-manager-5d77c4c564-kdn42"
Jul 01 14:46:07.791666 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:07.791646    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2451d25a-2e7e-46c3-ae65-cb5a8073f23b-cert\") pod \"kserve-controller-manager-5d77c4c564-kdn42\" (UID: \"2451d25a-2e7e-46c3-ae65-cb5a8073f23b\") " pod="kserve/kserve-controller-manager-5d77c4c564-kdn42"
Jul 01 14:46:07.904207 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:07.904167    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-5d77c4c564-kdn42"
Jul 01 14:46:08.041859 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:08.041826    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-5d77c4c564-kdn42"]
Jul 01 14:46:08.046250 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:46:08.046213    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2451d25a_2e7e_46c3_ae65_cb5a8073f23b.slice/crio-6dfc2cc5c93e31bed2c16af3b266d874b6026a81b65f6019fe92460e3f31c87f WatchSource:0}: Error finding container 6dfc2cc5c93e31bed2c16af3b266d874b6026a81b65f6019fe92460e3f31c87f: Status 404 returned error can't find the container with id 6dfc2cc5c93e31bed2c16af3b266d874b6026a81b65f6019fe92460e3f31c87f
Jul 01 14:46:08.727927 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:08.727886    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-5d77c4c564-kdn42" event={"ID":"2451d25a-2e7e-46c3-ae65-cb5a8073f23b","Type":"ContainerStarted","Data":"6dfc2cc5c93e31bed2c16af3b266d874b6026a81b65f6019fe92460e3f31c87f"}
Jul 01 14:46:11.743759 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:11.743719    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-vm2tg" event={"ID":"aa05ded8-91de-42fb-a8e4-62fc84f7498e","Type":"ContainerStarted","Data":"ca6ec623d5a6bf9ecca5cc6223d51d5bb6cf9ed2375b1dc5cd076cf222810b56"}
Jul 01 14:46:11.744181 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:11.743818    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-64568bcd49-vm2tg"
Jul 01 14:46:11.745299 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:11.745273    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-5d77c4c564-kdn42" event={"ID":"2451d25a-2e7e-46c3-ae65-cb5a8073f23b","Type":"ContainerStarted","Data":"df28b92df75c07a5f30cd2cbb343b04090d374e16e21382e38d6514a532ede53"}
Jul 01 14:46:11.745395 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:11.745321    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-5d77c4c564-kdn42"
Jul 01 14:46:11.770605 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:11.770541    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-64568bcd49-vm2tg" podStartSLOduration=0.719441699 podStartE2EDuration="4.770522566s" totalImagesPullingTime="4.051080867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:46:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:46:11.767890007 +0000 UTC m=+409.096462333" watchObservedRunningTime="2026-07-01 14:46:11.770522566 +0000 UTC m=+409.099094872"
Jul 01 14:46:11.788523 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:11.788462    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-5d77c4c564-kdn42" podStartSLOduration=2.324863305 podStartE2EDuration="5.78844638s" totalImagesPullingTime="3.463583075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:46:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:46:11.787591128 +0000 UTC m=+409.116163434" watchObservedRunningTime="2026-07-01 14:46:11.78844638 +0000 UTC m=+409.117018685"
Jul 01 14:46:17.752296 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:17.752257    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-64568bcd49-vm2tg"
Jul 01 14:46:42.754598 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:42.754556    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-5d77c4c564-kdn42"
Jul 01 14:46:43.221067 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:43.220974    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-5d77c4c564-kdn42"]
Jul 01 14:46:43.221225 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:43.221159    2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve/kserve-controller-manager-5d77c4c564-kdn42" podUID="2451d25a-2e7e-46c3-ae65-cb5a8073f23b" containerName="manager" containerID="cri-o://df28b92df75c07a5f30cd2cbb343b04090d374e16e21382e38d6514a532ede53" gracePeriod=10
Jul 01 14:46:43.252559 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:43.252513    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-5d77c4c564-thtll"]
Jul 01 14:46:43.255428 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:43.255403    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-5d77c4c564-thtll"
Jul 01 14:46:43.260789 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:43.260759    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-5d77c4c564-thtll"]
Jul 01 14:46:43.367929 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:43.367889    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6nxf\" (UniqueName: \"kubernetes.io/projected/210ae3d0-3cde-49d7-b680-1f19fac3b5a2-kube-api-access-c6nxf\") pod \"kserve-controller-manager-5d77c4c564-thtll\" (UID: \"210ae3d0-3cde-49d7-b680-1f19fac3b5a2\") " pod="kserve/kserve-controller-manager-5d77c4c564-thtll"
Jul 01 14:46:43.368113 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:43.368016    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/210ae3d0-3cde-49d7-b680-1f19fac3b5a2-cert\") pod \"kserve-controller-manager-5d77c4c564-thtll\" (UID: \"210ae3d0-3cde-49d7-b680-1f19fac3b5a2\") " pod="kserve/kserve-controller-manager-5d77c4c564-thtll"
Jul 01 14:46:43.462680 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:43.462654    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-5d77c4c564-kdn42"
Jul 01 14:46:43.468512 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:43.468489    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6nxf\" (UniqueName: \"kubernetes.io/projected/210ae3d0-3cde-49d7-b680-1f19fac3b5a2-kube-api-access-c6nxf\") pod \"kserve-controller-manager-5d77c4c564-thtll\" (UID: \"210ae3d0-3cde-49d7-b680-1f19fac3b5a2\") " pod="kserve/kserve-controller-manager-5d77c4c564-thtll"
Jul 01 14:46:43.468602 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:43.468540    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/210ae3d0-3cde-49d7-b680-1f19fac3b5a2-cert\") pod \"kserve-controller-manager-5d77c4c564-thtll\" (UID: \"210ae3d0-3cde-49d7-b680-1f19fac3b5a2\") " pod="kserve/kserve-controller-manager-5d77c4c564-thtll"
Jul 01 14:46:43.470812 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:43.470782    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/210ae3d0-3cde-49d7-b680-1f19fac3b5a2-cert\") pod \"kserve-controller-manager-5d77c4c564-thtll\" (UID: \"210ae3d0-3cde-49d7-b680-1f19fac3b5a2\") " pod="kserve/kserve-controller-manager-5d77c4c564-thtll"
Jul 01 14:46:43.477239 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:43.477211    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6nxf\" (UniqueName: \"kubernetes.io/projected/210ae3d0-3cde-49d7-b680-1f19fac3b5a2-kube-api-access-c6nxf\") pod \"kserve-controller-manager-5d77c4c564-thtll\" (UID: \"210ae3d0-3cde-49d7-b680-1f19fac3b5a2\") " pod="kserve/kserve-controller-manager-5d77c4c564-thtll"
Jul 01 14:46:43.569337 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:43.569305    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tllv7\" (UniqueName: \"kubernetes.io/projected/2451d25a-2e7e-46c3-ae65-cb5a8073f23b-kube-api-access-tllv7\") pod \"2451d25a-2e7e-46c3-ae65-cb5a8073f23b\" (UID: \"2451d25a-2e7e-46c3-ae65-cb5a8073f23b\") "
Jul 01 14:46:43.569337 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:43.569345    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2451d25a-2e7e-46c3-ae65-cb5a8073f23b-cert\") pod \"2451d25a-2e7e-46c3-ae65-cb5a8073f23b\" (UID: \"2451d25a-2e7e-46c3-ae65-cb5a8073f23b\") "
Jul 01 14:46:43.571575 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:43.571540    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2451d25a-2e7e-46c3-ae65-cb5a8073f23b-cert" (OuterVolumeSpecName: "cert") pod "2451d25a-2e7e-46c3-ae65-cb5a8073f23b" (UID: "2451d25a-2e7e-46c3-ae65-cb5a8073f23b"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 14:46:43.571731 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:43.571610    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2451d25a-2e7e-46c3-ae65-cb5a8073f23b-kube-api-access-tllv7" (OuterVolumeSpecName: "kube-api-access-tllv7") pod "2451d25a-2e7e-46c3-ae65-cb5a8073f23b" (UID: "2451d25a-2e7e-46c3-ae65-cb5a8073f23b"). InnerVolumeSpecName "kube-api-access-tllv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 14:46:43.599582 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:43.599545    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-5d77c4c564-thtll"
Jul 01 14:46:43.670555 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:43.670522    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tllv7\" (UniqueName: \"kubernetes.io/projected/2451d25a-2e7e-46c3-ae65-cb5a8073f23b-kube-api-access-tllv7\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:46:43.670555 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:43.670552    2551 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2451d25a-2e7e-46c3-ae65-cb5a8073f23b-cert\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:46:43.738776 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:43.738697    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-5d77c4c564-thtll"]
Jul 01 14:46:43.741901 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:46:43.741870    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod210ae3d0_3cde_49d7_b680_1f19fac3b5a2.slice/crio-b075115533dddef3195d6618d2db2625b472ee9c71856f559e9724316c503864 WatchSource:0}: Error finding container b075115533dddef3195d6618d2db2625b472ee9c71856f559e9724316c503864: Status 404 returned error can't find the container with id b075115533dddef3195d6618d2db2625b472ee9c71856f559e9724316c503864
Jul 01 14:46:43.873961 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:43.873869    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-5d77c4c564-thtll" event={"ID":"210ae3d0-3cde-49d7-b680-1f19fac3b5a2","Type":"ContainerStarted","Data":"b075115533dddef3195d6618d2db2625b472ee9c71856f559e9724316c503864"}
Jul 01 14:46:43.875069 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:43.875042    2551 generic.go:356] "Generic (PLEG): container finished" podID="2451d25a-2e7e-46c3-ae65-cb5a8073f23b" containerID="df28b92df75c07a5f30cd2cbb343b04090d374e16e21382e38d6514a532ede53" exitCode=0
Jul 01 14:46:43.875138 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:43.875093    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-5d77c4c564-kdn42"
Jul 01 14:46:43.875173 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:43.875124    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-5d77c4c564-kdn42" event={"ID":"2451d25a-2e7e-46c3-ae65-cb5a8073f23b","Type":"ContainerDied","Data":"df28b92df75c07a5f30cd2cbb343b04090d374e16e21382e38d6514a532ede53"}
Jul 01 14:46:43.875206 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:43.875180    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-5d77c4c564-kdn42" event={"ID":"2451d25a-2e7e-46c3-ae65-cb5a8073f23b","Type":"ContainerDied","Data":"6dfc2cc5c93e31bed2c16af3b266d874b6026a81b65f6019fe92460e3f31c87f"}
Jul 01 14:46:43.875238 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:43.875208    2551 scope.go:117] "RemoveContainer" containerID="df28b92df75c07a5f30cd2cbb343b04090d374e16e21382e38d6514a532ede53"
Jul 01 14:46:43.883826 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:43.883803    2551 scope.go:117] "RemoveContainer" containerID="df28b92df75c07a5f30cd2cbb343b04090d374e16e21382e38d6514a532ede53"
Jul 01 14:46:43.884136 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:46:43.884118    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df28b92df75c07a5f30cd2cbb343b04090d374e16e21382e38d6514a532ede53\": container with ID starting with df28b92df75c07a5f30cd2cbb343b04090d374e16e21382e38d6514a532ede53 not found: ID does not exist" containerID="df28b92df75c07a5f30cd2cbb343b04090d374e16e21382e38d6514a532ede53"
Jul 01 14:46:43.884201 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:43.884149    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df28b92df75c07a5f30cd2cbb343b04090d374e16e21382e38d6514a532ede53"} err="failed to get container status \"df28b92df75c07a5f30cd2cbb343b04090d374e16e21382e38d6514a532ede53\": rpc error: code = NotFound desc = could not find container \"df28b92df75c07a5f30cd2cbb343b04090d374e16e21382e38d6514a532ede53\": container with ID starting with df28b92df75c07a5f30cd2cbb343b04090d374e16e21382e38d6514a532ede53 not found: ID does not exist"
Jul 01 14:46:43.897346 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:43.897310    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-5d77c4c564-kdn42"]
Jul 01 14:46:43.903185 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:43.903158    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-5d77c4c564-kdn42"]
Jul 01 14:46:44.880800 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:44.880690    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-5d77c4c564-thtll" event={"ID":"210ae3d0-3cde-49d7-b680-1f19fac3b5a2","Type":"ContainerStarted","Data":"aabfa2f4615a16b8e2592322dd6f0b4e1689f494a955e7149cda00e1649a9eff"}
Jul 01 14:46:44.881239 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:44.880846    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-5d77c4c564-thtll"
Jul 01 14:46:44.899213 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:44.899158    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-5d77c4c564-thtll" podStartSLOduration=1.060722841 podStartE2EDuration="1.899135177s" totalImagesPullingTime="838.412336ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:46:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:46:44.898327073 +0000 UTC m=+442.226899380" watchObservedRunningTime="2026-07-01 14:46:44.899135177 +0000 UTC m=+442.227707484"
Jul 01 14:46:45.103452 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:46:45.103416    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2451d25a-2e7e-46c3-ae65-cb5a8073f23b" path="/var/lib/kubelet/pods/2451d25a-2e7e-46c3-ae65-cb5a8073f23b/volumes"
Jul 01 14:47:15.890525 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:15.890487    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-5d77c4c564-thtll"
Jul 01 14:47:17.177602 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:17.177564    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-fd65d7d6-5gw8k"]
Jul 01 14:47:17.178012 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:17.177885    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2451d25a-2e7e-46c3-ae65-cb5a8073f23b" containerName="manager"
Jul 01 14:47:17.178012 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:17.177898    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="2451d25a-2e7e-46c3-ae65-cb5a8073f23b" containerName="manager"
Jul 01 14:47:17.178012 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:17.177968    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="2451d25a-2e7e-46c3-ae65-cb5a8073f23b" containerName="manager"
Jul 01 14:47:17.179794 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:17.179772    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-fd65d7d6-5gw8k"
Jul 01 14:47:17.183836 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:17.183805    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\""
Jul 01 14:47:17.187935 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:17.187917    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-96bhw\""
Jul 01 14:47:17.199845 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:17.199814    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-fd65d7d6-5gw8k"]
Jul 01 14:47:17.202999 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:17.202973    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-84b54b7f97-47pwn"]
Jul 01 14:47:17.204983 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:17.204967    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-47pwn"
Jul 01 14:47:17.207492 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:17.207471    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-tvzg7\""
Jul 01 14:47:17.207649 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:17.207611    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
Jul 01 14:47:17.222429 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:17.222401    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-47pwn"]
Jul 01 14:47:17.253568 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:17.253533    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfcwh\" (UniqueName: \"kubernetes.io/projected/2c675763-aa41-4746-8cab-297bf7a82d9e-kube-api-access-sfcwh\") pod \"odh-model-controller-84b54b7f97-47pwn\" (UID: \"2c675763-aa41-4746-8cab-297bf7a82d9e\") " pod="kserve/odh-model-controller-84b54b7f97-47pwn"
Jul 01 14:47:17.253786 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:17.253579    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ttzd\" (UniqueName: \"kubernetes.io/projected/b345ac12-4ace-42ce-9631-3769287e311e-kube-api-access-4ttzd\") pod \"model-serving-api-fd65d7d6-5gw8k\" (UID: \"b345ac12-4ace-42ce-9631-3769287e311e\") " pod="kserve/model-serving-api-fd65d7d6-5gw8k"
Jul 01 14:47:17.253786 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:17.253613    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c675763-aa41-4746-8cab-297bf7a82d9e-cert\") pod \"odh-model-controller-84b54b7f97-47pwn\" (UID: \"2c675763-aa41-4746-8cab-297bf7a82d9e\") " pod="kserve/odh-model-controller-84b54b7f97-47pwn"
Jul 01 14:47:17.253786 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:17.253658    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b345ac12-4ace-42ce-9631-3769287e311e-tls-certs\") pod \"model-serving-api-fd65d7d6-5gw8k\" (UID: \"b345ac12-4ace-42ce-9631-3769287e311e\") " pod="kserve/model-serving-api-fd65d7d6-5gw8k"
Jul 01 14:47:17.354418 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:17.354385    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ttzd\" (UniqueName: \"kubernetes.io/projected/b345ac12-4ace-42ce-9631-3769287e311e-kube-api-access-4ttzd\") pod \"model-serving-api-fd65d7d6-5gw8k\" (UID: \"b345ac12-4ace-42ce-9631-3769287e311e\") " pod="kserve/model-serving-api-fd65d7d6-5gw8k"
Jul 01 14:47:17.354648 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:17.354451    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c675763-aa41-4746-8cab-297bf7a82d9e-cert\") pod \"odh-model-controller-84b54b7f97-47pwn\" (UID: \"2c675763-aa41-4746-8cab-297bf7a82d9e\") " pod="kserve/odh-model-controller-84b54b7f97-47pwn"
Jul 01 14:47:17.354648 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:17.354482    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b345ac12-4ace-42ce-9631-3769287e311e-tls-certs\") pod \"model-serving-api-fd65d7d6-5gw8k\" (UID: \"b345ac12-4ace-42ce-9631-3769287e311e\") " pod="kserve/model-serving-api-fd65d7d6-5gw8k"
Jul 01 14:47:17.354648 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:17.354527    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfcwh\" (UniqueName: \"kubernetes.io/projected/2c675763-aa41-4746-8cab-297bf7a82d9e-kube-api-access-sfcwh\") pod \"odh-model-controller-84b54b7f97-47pwn\" (UID: \"2c675763-aa41-4746-8cab-297bf7a82d9e\") " pod="kserve/odh-model-controller-84b54b7f97-47pwn"
Jul 01 14:47:17.354648 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:47:17.354600    2551 secret.go:189] Couldn't get secret kserve/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 01 14:47:17.354808 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:47:17.354692    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c675763-aa41-4746-8cab-297bf7a82d9e-cert podName:2c675763-aa41-4746-8cab-297bf7a82d9e nodeName:}" failed. No retries permitted until 2026-07-01 14:47:17.85467559 +0000 UTC m=+475.183247874 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2c675763-aa41-4746-8cab-297bf7a82d9e-cert") pod "odh-model-controller-84b54b7f97-47pwn" (UID: "2c675763-aa41-4746-8cab-297bf7a82d9e") : secret "odh-model-controller-webhook-cert" not found
Jul 01 14:47:17.354808 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:47:17.354610    2551 secret.go:189] Couldn't get secret kserve/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 01 14:47:17.354808 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:47:17.354762    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b345ac12-4ace-42ce-9631-3769287e311e-tls-certs podName:b345ac12-4ace-42ce-9631-3769287e311e nodeName:}" failed. No retries permitted until 2026-07-01 14:47:17.85474507 +0000 UTC m=+475.183317354 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/b345ac12-4ace-42ce-9631-3769287e311e-tls-certs") pod "model-serving-api-fd65d7d6-5gw8k" (UID: "b345ac12-4ace-42ce-9631-3769287e311e") : secret "model-serving-api-tls" not found
Jul 01 14:47:17.364320 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:17.364292    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfcwh\" (UniqueName: \"kubernetes.io/projected/2c675763-aa41-4746-8cab-297bf7a82d9e-kube-api-access-sfcwh\") pod \"odh-model-controller-84b54b7f97-47pwn\" (UID: \"2c675763-aa41-4746-8cab-297bf7a82d9e\") " pod="kserve/odh-model-controller-84b54b7f97-47pwn"
Jul 01 14:47:17.366073 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:17.366054    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ttzd\" (UniqueName: \"kubernetes.io/projected/b345ac12-4ace-42ce-9631-3769287e311e-kube-api-access-4ttzd\") pod \"model-serving-api-fd65d7d6-5gw8k\" (UID: \"b345ac12-4ace-42ce-9631-3769287e311e\") " pod="kserve/model-serving-api-fd65d7d6-5gw8k"
Jul 01 14:47:17.858665 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:17.858599    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b345ac12-4ace-42ce-9631-3769287e311e-tls-certs\") pod \"model-serving-api-fd65d7d6-5gw8k\" (UID: \"b345ac12-4ace-42ce-9631-3769287e311e\") " pod="kserve/model-serving-api-fd65d7d6-5gw8k"
Jul 01 14:47:17.858842 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:17.858727    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c675763-aa41-4746-8cab-297bf7a82d9e-cert\") pod \"odh-model-controller-84b54b7f97-47pwn\" (UID: \"2c675763-aa41-4746-8cab-297bf7a82d9e\") " pod="kserve/odh-model-controller-84b54b7f97-47pwn"
Jul 01 14:47:17.861206 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:17.861169    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b345ac12-4ace-42ce-9631-3769287e311e-tls-certs\") pod \"model-serving-api-fd65d7d6-5gw8k\" (UID: \"b345ac12-4ace-42ce-9631-3769287e311e\") " pod="kserve/model-serving-api-fd65d7d6-5gw8k"
Jul 01 14:47:17.861326 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:17.861169    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c675763-aa41-4746-8cab-297bf7a82d9e-cert\") pod \"odh-model-controller-84b54b7f97-47pwn\" (UID: \"2c675763-aa41-4746-8cab-297bf7a82d9e\") " pod="kserve/odh-model-controller-84b54b7f97-47pwn"
Jul 01 14:47:18.089913 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:18.089850    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-fd65d7d6-5gw8k"
Jul 01 14:47:18.116046 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:18.115966    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-47pwn"
Jul 01 14:47:18.234537 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:18.234512    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-fd65d7d6-5gw8k"]
Jul 01 14:47:18.237174 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:47:18.237143    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb345ac12_4ace_42ce_9631_3769287e311e.slice/crio-51da07b9604a0220f7150f6f4375ccda4d918dccffc05d4139976606b6207e1e WatchSource:0}: Error finding container 51da07b9604a0220f7150f6f4375ccda4d918dccffc05d4139976606b6207e1e: Status 404 returned error can't find the container with id 51da07b9604a0220f7150f6f4375ccda4d918dccffc05d4139976606b6207e1e
Jul 01 14:47:18.266584 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:18.266550    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-47pwn"]
Jul 01 14:47:18.269397 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:47:18.269366    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c675763_aa41_4746_8cab_297bf7a82d9e.slice/crio-6de77383b4b85290d7551958d0f05e0346ffde0b8615e91172ee475cd715ea76 WatchSource:0}: Error finding container 6de77383b4b85290d7551958d0f05e0346ffde0b8615e91172ee475cd715ea76: Status 404 returned error can't find the container with id 6de77383b4b85290d7551958d0f05e0346ffde0b8615e91172ee475cd715ea76
Jul 01 14:47:19.031053 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:19.030953    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-47pwn" event={"ID":"2c675763-aa41-4746-8cab-297bf7a82d9e","Type":"ContainerStarted","Data":"6de77383b4b85290d7551958d0f05e0346ffde0b8615e91172ee475cd715ea76"}
Jul 01 14:47:19.039208 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:19.039127    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-fd65d7d6-5gw8k" event={"ID":"b345ac12-4ace-42ce-9631-3769287e311e","Type":"ContainerStarted","Data":"51da07b9604a0220f7150f6f4375ccda4d918dccffc05d4139976606b6207e1e"}
Jul 01 14:47:22.055850 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:22.055801    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-47pwn" event={"ID":"2c675763-aa41-4746-8cab-297bf7a82d9e","Type":"ContainerStarted","Data":"0c722496adb0f9da4ad2d2aa3bdda4a5cf2fde88593719f4e42fb676acee15ed"}
Jul 01 14:47:22.056361 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:22.055909    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-84b54b7f97-47pwn"
Jul 01 14:47:22.057562 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:22.057533    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-fd65d7d6-5gw8k" event={"ID":"b345ac12-4ace-42ce-9631-3769287e311e","Type":"ContainerStarted","Data":"4ae82a8898e8d59b4deebe61495304c596c9fc744f1f2dbb5b0e32cd4bdf32a6"}
Jul 01 14:47:22.057730 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:22.057686    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-fd65d7d6-5gw8k"
Jul 01 14:47:22.073596 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:22.073539    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-84b54b7f97-47pwn" podStartSLOduration=1.370862298 podStartE2EDuration="5.073523273s" totalImagesPullingTime="3.702660975s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:47:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:47:22.072245398 +0000 UTC m=+479.400817708" watchObservedRunningTime="2026-07-01 14:47:22.073523273 +0000 UTC m=+479.402095580"
Jul 01 14:47:22.088171 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:22.088113    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-fd65d7d6-5gw8k" podStartSLOduration=1.3578740919999999 podStartE2EDuration="5.088095394s" totalImagesPullingTime="3.730221302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:47:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:47:22.087170392 +0000 UTC m=+479.415742698" watchObservedRunningTime="2026-07-01 14:47:22.088095394 +0000 UTC m=+479.416667699"
Jul 01 14:47:33.065045 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:33.065010    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-84b54b7f97-47pwn"
Jul 01 14:47:33.066677 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:47:33.066658    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-fd65d7d6-5gw8k"
Jul 01 14:49:04.844089 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:04.844054    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g"]
Jul 01 14:49:04.846475 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:04.846458    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g"
Jul 01 14:49:04.849916 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:04.849887    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-680c6-kube-rbac-proxy-sar-config\""
Jul 01 14:49:04.850058 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:04.849975    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-680c6-predictor-serving-cert\""
Jul 01 14:49:04.850058 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:04.849985    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 01 14:49:04.850827 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:04.850810    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 01 14:49:04.850909 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:04.850810    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-9cpsj\""
Jul 01 14:49:04.864437 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:04.864408    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g"]
Jul 01 14:49:04.886017 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:04.885988    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxsw8\" (UniqueName: \"kubernetes.io/projected/0d2bfcc6-62bf-4850-b9fd-30ac703445bc-kube-api-access-jxsw8\") pod \"success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g\" (UID: \"0d2bfcc6-62bf-4850-b9fd-30ac703445bc\") " pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g"
Jul 01 14:49:04.886191 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:04.886023    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0d2bfcc6-62bf-4850-b9fd-30ac703445bc-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g\" (UID: \"0d2bfcc6-62bf-4850-b9fd-30ac703445bc\") " pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g"
Jul 01 14:49:04.886191 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:04.886093    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0d2bfcc6-62bf-4850-b9fd-30ac703445bc-proxy-tls\") pod \"success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g\" (UID: \"0d2bfcc6-62bf-4850-b9fd-30ac703445bc\") " pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g"
Jul 01 14:49:04.987054 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:04.987015    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxsw8\" (UniqueName: \"kubernetes.io/projected/0d2bfcc6-62bf-4850-b9fd-30ac703445bc-kube-api-access-jxsw8\") pod \"success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g\" (UID: \"0d2bfcc6-62bf-4850-b9fd-30ac703445bc\") " pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g"
Jul 01 14:49:04.987251 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:04.987070    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0d2bfcc6-62bf-4850-b9fd-30ac703445bc-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g\" (UID: \"0d2bfcc6-62bf-4850-b9fd-30ac703445bc\") " pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g"
Jul 01 14:49:04.987251 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:04.987136    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0d2bfcc6-62bf-4850-b9fd-30ac703445bc-proxy-tls\") pod \"success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g\" (UID: \"0d2bfcc6-62bf-4850-b9fd-30ac703445bc\") " pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g"
Jul 01 14:49:04.987372 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:49:04.987320    2551 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-680c6-predictor-serving-cert: secret "success-200-isvc-680c6-predictor-serving-cert" not found
Jul 01 14:49:04.987425 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:49:04.987414    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d2bfcc6-62bf-4850-b9fd-30ac703445bc-proxy-tls podName:0d2bfcc6-62bf-4850-b9fd-30ac703445bc nodeName:}" failed. No retries permitted until 2026-07-01 14:49:05.487386182 +0000 UTC m=+582.815958483 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/0d2bfcc6-62bf-4850-b9fd-30ac703445bc-proxy-tls") pod "success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g" (UID: "0d2bfcc6-62bf-4850-b9fd-30ac703445bc") : secret "success-200-isvc-680c6-predictor-serving-cert" not found
Jul 01 14:49:04.987699 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:04.987676    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0d2bfcc6-62bf-4850-b9fd-30ac703445bc-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g\" (UID: \"0d2bfcc6-62bf-4850-b9fd-30ac703445bc\") " pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g"
Jul 01 14:49:04.996413 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:04.996392    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxsw8\" (UniqueName: \"kubernetes.io/projected/0d2bfcc6-62bf-4850-b9fd-30ac703445bc-kube-api-access-jxsw8\") pod \"success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g\" (UID: \"0d2bfcc6-62bf-4850-b9fd-30ac703445bc\") " pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g"
Jul 01 14:49:05.241701 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:05.241667    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s"]
Jul 01 14:49:05.244353 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:05.244330    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s"
Jul 01 14:49:05.247079 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:05.247053    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 01 14:49:05.247219 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:05.247125    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\""
Jul 01 14:49:05.247219 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:05.247167    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-predictor-serving-cert\""
Jul 01 14:49:05.247219 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:05.247184    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 01 14:49:05.257062 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:05.257037    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s"]
Jul 01 14:49:05.289848 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:05.289810    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s\" (UID: \"01046ee7-828b-4082-8f82-a9fe6bfa9ebc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s"
Jul 01 14:49:05.290113 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:05.289864    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s\" (UID: \"01046ee7-828b-4082-8f82-a9fe6bfa9ebc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s"
Jul 01 14:49:05.290113 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:05.289972    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s\" (UID: \"01046ee7-828b-4082-8f82-a9fe6bfa9ebc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s"
Jul 01 14:49:05.290113 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:05.290038    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmfc5\" (UniqueName: \"kubernetes.io/projected/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-kube-api-access-mmfc5\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s\" (UID: \"01046ee7-828b-4082-8f82-a9fe6bfa9ebc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s"
Jul 01 14:49:05.290113 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:05.290083    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s\" (UID: \"01046ee7-828b-4082-8f82-a9fe6bfa9ebc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s"
Jul 01 14:49:05.390892 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:05.390857    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s\" (UID: \"01046ee7-828b-4082-8f82-a9fe6bfa9ebc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s"
Jul 01 14:49:05.391084 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:05.390916    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s\" (UID: \"01046ee7-828b-4082-8f82-a9fe6bfa9ebc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s"
Jul 01 14:49:05.391084 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:05.390981    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s\" (UID: \"01046ee7-828b-4082-8f82-a9fe6bfa9ebc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s"
Jul 01 14:49:05.391084 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:05.391022    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmfc5\" (UniqueName: \"kubernetes.io/projected/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-kube-api-access-mmfc5\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s\" (UID: \"01046ee7-828b-4082-8f82-a9fe6bfa9ebc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s"
Jul 01 14:49:05.391084 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:49:05.391035    2551 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-serving-cert: secret "isvc-sklearn-graph-2-predictor-serving-cert" not found
Jul 01 14:49:05.391084 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:05.391056    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s\" (UID: \"01046ee7-828b-4082-8f82-a9fe6bfa9ebc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s"
Jul 01 14:49:05.391342 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:49:05.391118    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-proxy-tls podName:01046ee7-828b-4082-8f82-a9fe6bfa9ebc nodeName:}" failed. No retries permitted until 2026-07-01 14:49:05.891097798 +0000 UTC m=+583.219670106 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-proxy-tls") pod "isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s" (UID: "01046ee7-828b-4082-8f82-a9fe6bfa9ebc") : secret "isvc-sklearn-graph-2-predictor-serving-cert" not found
Jul 01 14:49:05.391977 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:05.391412    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s\" (UID: \"01046ee7-828b-4082-8f82-a9fe6bfa9ebc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s"
Jul 01 14:49:05.391977 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:05.391925    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s\" (UID: \"01046ee7-828b-4082-8f82-a9fe6bfa9ebc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s"
Jul 01 14:49:05.391977 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:05.391944    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s\" (UID: \"01046ee7-828b-4082-8f82-a9fe6bfa9ebc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s"
Jul 01 14:49:05.401307 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:05.401278    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmfc5\" (UniqueName: \"kubernetes.io/projected/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-kube-api-access-mmfc5\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s\" (UID: \"01046ee7-828b-4082-8f82-a9fe6bfa9ebc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s"
Jul 01 14:49:05.491861 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:05.491759    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0d2bfcc6-62bf-4850-b9fd-30ac703445bc-proxy-tls\") pod \"success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g\" (UID: \"0d2bfcc6-62bf-4850-b9fd-30ac703445bc\") " pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g"
Jul 01 14:49:05.494669 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:05.494639    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0d2bfcc6-62bf-4850-b9fd-30ac703445bc-proxy-tls\") pod \"success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g\" (UID: \"0d2bfcc6-62bf-4850-b9fd-30ac703445bc\") " pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g"
Jul 01 14:49:05.756774 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:05.756676    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g"
Jul 01 14:49:05.895392 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:05.895354    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s\" (UID: \"01046ee7-828b-4082-8f82-a9fe6bfa9ebc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s"
Jul 01 14:49:05.898213 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:05.898177    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g"]
Jul 01 14:49:05.899314 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:05.899297    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s\" (UID: \"01046ee7-828b-4082-8f82-a9fe6bfa9ebc\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s"
Jul 01 14:49:05.902268 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:49:05.902239    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d2bfcc6_62bf_4850_b9fd_30ac703445bc.slice/crio-54211983bfb1d14e2fff9a8e639dc994e8f2f4fcda8241caf727b2cee340a5b0 WatchSource:0}: Error finding container 54211983bfb1d14e2fff9a8e639dc994e8f2f4fcda8241caf727b2cee340a5b0: Status 404 returned error can't find the container with id 54211983bfb1d14e2fff9a8e639dc994e8f2f4fcda8241caf727b2cee340a5b0
Jul 01 14:49:06.155814 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:06.155719    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s"
Jul 01 14:49:06.292009 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:06.291977    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s"]
Jul 01 14:49:06.294310 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:49:06.294276    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01046ee7_828b_4082_8f82_a9fe6bfa9ebc.slice/crio-1b817432af365e796fa2b790781183ddc5fb5de0b2579dd3409ff24489d733e4 WatchSource:0}: Error finding container 1b817432af365e796fa2b790781183ddc5fb5de0b2579dd3409ff24489d733e4: Status 404 returned error can't find the container with id 1b817432af365e796fa2b790781183ddc5fb5de0b2579dd3409ff24489d733e4
Jul 01 14:49:06.482263 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:06.482226    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g" event={"ID":"0d2bfcc6-62bf-4850-b9fd-30ac703445bc","Type":"ContainerStarted","Data":"54211983bfb1d14e2fff9a8e639dc994e8f2f4fcda8241caf727b2cee340a5b0"}
Jul 01 14:49:06.483342 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:06.483315    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s" event={"ID":"01046ee7-828b-4082-8f82-a9fe6bfa9ebc","Type":"ContainerStarted","Data":"1b817432af365e796fa2b790781183ddc5fb5de0b2579dd3409ff24489d733e4"}
Jul 01 14:49:19.558145 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:19.558111    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s" event={"ID":"01046ee7-828b-4082-8f82-a9fe6bfa9ebc","Type":"ContainerStarted","Data":"231a64244f0ff9d223b6b4b5857ccba5fc218f7d609b178d573706dc5bbc8607"}
Jul 01 14:49:20.566126 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:20.566079    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g" event={"ID":"0d2bfcc6-62bf-4850-b9fd-30ac703445bc","Type":"ContainerStarted","Data":"de0db827a3a0da63d9b7b627027ce559cd08612fa479847376ca8c10d7589ead"}
Jul 01 14:49:22.578052 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:22.578011    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g" event={"ID":"0d2bfcc6-62bf-4850-b9fd-30ac703445bc","Type":"ContainerStarted","Data":"83ce75fd7f77b60c0e05a9cc33ccffd8d62eebc62965ff3cc4e2b8b4c862705e"}
Jul 01 14:49:22.578558 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:22.578220    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g"
Jul 01 14:49:22.578558 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:22.578351    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g"
Jul 01 14:49:22.579748 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:22.579721    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g" podUID="0d2bfcc6-62bf-4850-b9fd-30ac703445bc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 01 14:49:22.597213 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:22.597148    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g" podStartSLOduration=2.533087285 podStartE2EDuration="18.597129325s" totalImagesPullingTime="16.06404204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:49:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:49:22.595249207 +0000 UTC m=+599.923821515" watchObservedRunningTime="2026-07-01 14:49:22.597129325 +0000 UTC m=+599.925701631"
Jul 01 14:49:23.068934 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:23.068899    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/ovn-acl-logging/0.log"
Jul 01 14:49:23.069412 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:23.069396    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/ovn-acl-logging/0.log"
Jul 01 14:49:23.583353 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:23.583318    2551 generic.go:356] "Generic (PLEG): container finished" podID="01046ee7-828b-4082-8f82-a9fe6bfa9ebc" containerID="231a64244f0ff9d223b6b4b5857ccba5fc218f7d609b178d573706dc5bbc8607" exitCode=0
Jul 01 14:49:23.583800 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:23.583393    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s" event={"ID":"01046ee7-828b-4082-8f82-a9fe6bfa9ebc","Type":"ContainerDied","Data":"231a64244f0ff9d223b6b4b5857ccba5fc218f7d609b178d573706dc5bbc8607"}
Jul 01 14:49:23.583800 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:23.583683    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g" podUID="0d2bfcc6-62bf-4850-b9fd-30ac703445bc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 01 14:49:28.589474 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:28.589445    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g"
Jul 01 14:49:28.628968 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:28.590026    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g" podUID="0d2bfcc6-62bf-4850-b9fd-30ac703445bc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 01 14:49:29.614463 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:29.614426    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s" event={"ID":"01046ee7-828b-4082-8f82-a9fe6bfa9ebc","Type":"ContainerStarted","Data":"6ed8b3d3f2852791b1e1ee96b7503c381fece3f84a207ed1eb9e56ded16202a0"}
Jul 01 14:49:29.614936 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:29.614491    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s" event={"ID":"01046ee7-828b-4082-8f82-a9fe6bfa9ebc","Type":"ContainerStarted","Data":"6e38051030a08853965de8939cfec3a35c3c14d2dbed41ce1aad2abc04a3830d"}
Jul 01 14:49:29.614936 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:29.614767    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s"
Jul 01 14:49:29.633042 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:29.632965    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s" podStartSLOduration=1.968536517 podStartE2EDuration="24.632943192s" totalImagesPullingTime="18.980276072s" totalInitContainerRuntime="3.684130603s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:49:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:49:29.632217528 +0000 UTC m=+606.960789873" watchObservedRunningTime="2026-07-01 14:49:29.632943192 +0000 UTC m=+606.961515501"
Jul 01 14:49:30.618856 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:30.618824    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s"
Jul 01 14:49:30.620475 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:30.620444    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s" podUID="01046ee7-828b-4082-8f82-a9fe6bfa9ebc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Jul 01 14:49:31.623220 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:31.623179    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s" podUID="01046ee7-828b-4082-8f82-a9fe6bfa9ebc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Jul 01 14:49:36.629063 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:36.629033    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s"
Jul 01 14:49:36.629610 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:36.629585    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s" podUID="01046ee7-828b-4082-8f82-a9fe6bfa9ebc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Jul 01 14:49:38.590458 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:38.590409    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g" podUID="0d2bfcc6-62bf-4850-b9fd-30ac703445bc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 01 14:49:46.629896 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:46.629805    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s" podUID="01046ee7-828b-4082-8f82-a9fe6bfa9ebc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Jul 01 14:49:48.591108 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:48.591063    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g" podUID="0d2bfcc6-62bf-4850-b9fd-30ac703445bc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 01 14:49:56.630228 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:56.630190    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s" podUID="01046ee7-828b-4082-8f82-a9fe6bfa9ebc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Jul 01 14:49:58.590835 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:49:58.590780    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g" podUID="0d2bfcc6-62bf-4850-b9fd-30ac703445bc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 01 14:50:06.629869 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:06.629832    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s" podUID="01046ee7-828b-4082-8f82-a9fe6bfa9ebc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Jul 01 14:50:08.590797 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:08.590764    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g"
Jul 01 14:50:16.630040 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:16.629998    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s" podUID="01046ee7-828b-4082-8f82-a9fe6bfa9ebc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Jul 01 14:50:26.629876 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:26.629837    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s" podUID="01046ee7-828b-4082-8f82-a9fe6bfa9ebc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Jul 01 14:50:36.629766 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:36.629724    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s" podUID="01046ee7-828b-4082-8f82-a9fe6bfa9ebc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Jul 01 14:50:40.098983 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:40.098946    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s" podUID="01046ee7-828b-4082-8f82-a9fe6bfa9ebc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Jul 01 14:50:44.976461 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:44.976427    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg"]
Jul 01 14:50:44.978885 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:44.978863    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg"
Jul 01 14:50:44.982411 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:44.982386    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-3297d-predictor-serving-cert\""
Jul 01 14:50:44.986030 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:44.986012    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-3297d-kube-rbac-proxy-sar-config\""
Jul 01 14:50:45.003429 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:45.003394    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg"]
Jul 01 14:50:45.034040 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:45.034002    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/124c2e52-d9a7-48ce-86e5-0e24d7c801b1-proxy-tls\") pod \"success-200-isvc-3297d-predictor-5494fbf467-jmcgg\" (UID: \"124c2e52-d9a7-48ce-86e5-0e24d7c801b1\") " pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg"
Jul 01 14:50:45.034229 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:45.034067    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx65b\" (UniqueName: \"kubernetes.io/projected/124c2e52-d9a7-48ce-86e5-0e24d7c801b1-kube-api-access-jx65b\") pod \"success-200-isvc-3297d-predictor-5494fbf467-jmcgg\" (UID: \"124c2e52-d9a7-48ce-86e5-0e24d7c801b1\") " pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg"
Jul 01 14:50:45.034229 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:45.034102    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/124c2e52-d9a7-48ce-86e5-0e24d7c801b1-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-3297d-predictor-5494fbf467-jmcgg\" (UID: \"124c2e52-d9a7-48ce-86e5-0e24d7c801b1\") " pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg"
Jul 01 14:50:45.123350 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:45.123298    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g"]
Jul 01 14:50:45.123734 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:45.123699    2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g" podUID="0d2bfcc6-62bf-4850-b9fd-30ac703445bc" containerName="kserve-container" containerID="cri-o://de0db827a3a0da63d9b7b627027ce559cd08612fa479847376ca8c10d7589ead" gracePeriod=30
Jul 01 14:50:45.126321 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:45.124013    2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g" podUID="0d2bfcc6-62bf-4850-b9fd-30ac703445bc" containerName="kube-rbac-proxy" containerID="cri-o://83ce75fd7f77b60c0e05a9cc33ccffd8d62eebc62965ff3cc4e2b8b4c862705e" gracePeriod=30
Jul 01 14:50:45.134780 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:45.134751    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/124c2e52-d9a7-48ce-86e5-0e24d7c801b1-proxy-tls\") pod \"success-200-isvc-3297d-predictor-5494fbf467-jmcgg\" (UID: \"124c2e52-d9a7-48ce-86e5-0e24d7c801b1\") " pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg"
Jul 01 14:50:45.134916 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:45.134809    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx65b\" (UniqueName: \"kubernetes.io/projected/124c2e52-d9a7-48ce-86e5-0e24d7c801b1-kube-api-access-jx65b\") pod \"success-200-isvc-3297d-predictor-5494fbf467-jmcgg\" (UID: \"124c2e52-d9a7-48ce-86e5-0e24d7c801b1\") " pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg"
Jul 01 14:50:45.134916 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:45.134839    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/124c2e52-d9a7-48ce-86e5-0e24d7c801b1-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-3297d-predictor-5494fbf467-jmcgg\" (UID: \"124c2e52-d9a7-48ce-86e5-0e24d7c801b1\") " pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg"
Jul 01 14:50:45.134916 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:50:45.134867    2551 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-3297d-predictor-serving-cert: secret "success-200-isvc-3297d-predictor-serving-cert" not found
Jul 01 14:50:45.135660 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:50:45.134950    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/124c2e52-d9a7-48ce-86e5-0e24d7c801b1-proxy-tls podName:124c2e52-d9a7-48ce-86e5-0e24d7c801b1 nodeName:}" failed. No retries permitted until 2026-07-01 14:50:45.634927617 +0000 UTC m=+682.963499903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/124c2e52-d9a7-48ce-86e5-0e24d7c801b1-proxy-tls") pod "success-200-isvc-3297d-predictor-5494fbf467-jmcgg" (UID: "124c2e52-d9a7-48ce-86e5-0e24d7c801b1") : secret "success-200-isvc-3297d-predictor-serving-cert" not found
Jul 01 14:50:45.135768 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:45.135675    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/124c2e52-d9a7-48ce-86e5-0e24d7c801b1-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-3297d-predictor-5494fbf467-jmcgg\" (UID: \"124c2e52-d9a7-48ce-86e5-0e24d7c801b1\") " pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg"
Jul 01 14:50:45.145507 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:45.145470    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx65b\" (UniqueName: \"kubernetes.io/projected/124c2e52-d9a7-48ce-86e5-0e24d7c801b1-kube-api-access-jx65b\") pod \"success-200-isvc-3297d-predictor-5494fbf467-jmcgg\" (UID: \"124c2e52-d9a7-48ce-86e5-0e24d7c801b1\") " pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg"
Jul 01 14:50:45.638251 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:45.638161    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/124c2e52-d9a7-48ce-86e5-0e24d7c801b1-proxy-tls\") pod \"success-200-isvc-3297d-predictor-5494fbf467-jmcgg\" (UID: \"124c2e52-d9a7-48ce-86e5-0e24d7c801b1\") " pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg"
Jul 01 14:50:45.640617 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:45.640597    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/124c2e52-d9a7-48ce-86e5-0e24d7c801b1-proxy-tls\") pod \"success-200-isvc-3297d-predictor-5494fbf467-jmcgg\" (UID: \"124c2e52-d9a7-48ce-86e5-0e24d7c801b1\") " pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg"
Jul 01 14:50:45.888865 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:45.888766    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg"
Jul 01 14:50:45.945178 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:45.945009    2551 generic.go:356] "Generic (PLEG): container finished" podID="0d2bfcc6-62bf-4850-b9fd-30ac703445bc" containerID="83ce75fd7f77b60c0e05a9cc33ccffd8d62eebc62965ff3cc4e2b8b4c862705e" exitCode=2
Jul 01 14:50:45.945178 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:45.945068    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g" event={"ID":"0d2bfcc6-62bf-4850-b9fd-30ac703445bc","Type":"ContainerDied","Data":"83ce75fd7f77b60c0e05a9cc33ccffd8d62eebc62965ff3cc4e2b8b4c862705e"}
Jul 01 14:50:46.031049 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:46.031013    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg"]
Jul 01 14:50:46.033550 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:50:46.033524    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod124c2e52_d9a7_48ce_86e5_0e24d7c801b1.slice/crio-c96cc05553633b4bf8f09e9d4819f75e8be435e4aa8722855df3e25d76ff27b5 WatchSource:0}: Error finding container c96cc05553633b4bf8f09e9d4819f75e8be435e4aa8722855df3e25d76ff27b5: Status 404 returned error can't find the container with id c96cc05553633b4bf8f09e9d4819f75e8be435e4aa8722855df3e25d76ff27b5
Jul 01 14:50:46.951736 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:46.951704    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg" event={"ID":"124c2e52-d9a7-48ce-86e5-0e24d7c801b1","Type":"ContainerStarted","Data":"e624e0a11ae75c90c83b6d0dc6043ecb872d8bf16f8f595cab6ee61c29e0f408"}
Jul 01 14:50:46.951937 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:46.951759    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg" event={"ID":"124c2e52-d9a7-48ce-86e5-0e24d7c801b1","Type":"ContainerStarted","Data":"925b52940735f4fd596c2dc655b97cee159cdc820c7b15b3714c1f200c80e4a6"}
Jul 01 14:50:46.951937 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:46.951783    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg" event={"ID":"124c2e52-d9a7-48ce-86e5-0e24d7c801b1","Type":"ContainerStarted","Data":"c96cc05553633b4bf8f09e9d4819f75e8be435e4aa8722855df3e25d76ff27b5"}
Jul 01 14:50:46.951937 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:46.951852    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg"
Jul 01 14:50:46.972496 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:46.972440    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg" podStartSLOduration=2.972422371 podStartE2EDuration="2.972422371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:50:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:50:46.971103195 +0000 UTC m=+684.299675515" watchObservedRunningTime="2026-07-01 14:50:46.972422371 +0000 UTC m=+684.300994677"
Jul 01 14:50:47.957821 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:47.957780    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg"
Jul 01 14:50:47.959320 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:47.959288    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg" podUID="124c2e52-d9a7-48ce-86e5-0e24d7c801b1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Jul 01 14:50:48.588966 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:48.588937    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g"
Jul 01 14:50:48.660478 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:48.660381    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxsw8\" (UniqueName: \"kubernetes.io/projected/0d2bfcc6-62bf-4850-b9fd-30ac703445bc-kube-api-access-jxsw8\") pod \"0d2bfcc6-62bf-4850-b9fd-30ac703445bc\" (UID: \"0d2bfcc6-62bf-4850-b9fd-30ac703445bc\") "
Jul 01 14:50:48.660478 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:48.660424    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0d2bfcc6-62bf-4850-b9fd-30ac703445bc-kube-rbac-proxy-sar-config\") pod \"0d2bfcc6-62bf-4850-b9fd-30ac703445bc\" (UID: \"0d2bfcc6-62bf-4850-b9fd-30ac703445bc\") "
Jul 01 14:50:48.660478 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:48.660478    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0d2bfcc6-62bf-4850-b9fd-30ac703445bc-proxy-tls\") pod \"0d2bfcc6-62bf-4850-b9fd-30ac703445bc\" (UID: \"0d2bfcc6-62bf-4850-b9fd-30ac703445bc\") "
Jul 01 14:50:48.660883 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:48.660854    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d2bfcc6-62bf-4850-b9fd-30ac703445bc-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "0d2bfcc6-62bf-4850-b9fd-30ac703445bc" (UID: "0d2bfcc6-62bf-4850-b9fd-30ac703445bc"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:50:48.662810 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:48.662792    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d2bfcc6-62bf-4850-b9fd-30ac703445bc-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "0d2bfcc6-62bf-4850-b9fd-30ac703445bc" (UID: "0d2bfcc6-62bf-4850-b9fd-30ac703445bc"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 14:50:48.662859 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:48.662810    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d2bfcc6-62bf-4850-b9fd-30ac703445bc-kube-api-access-jxsw8" (OuterVolumeSpecName: "kube-api-access-jxsw8") pod "0d2bfcc6-62bf-4850-b9fd-30ac703445bc" (UID: "0d2bfcc6-62bf-4850-b9fd-30ac703445bc"). InnerVolumeSpecName "kube-api-access-jxsw8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 14:50:48.761423 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:48.761382    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxsw8\" (UniqueName: \"kubernetes.io/projected/0d2bfcc6-62bf-4850-b9fd-30ac703445bc-kube-api-access-jxsw8\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:50:48.761423 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:48.761415    2551 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0d2bfcc6-62bf-4850-b9fd-30ac703445bc-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:50:48.761423 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:48.761425    2551 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0d2bfcc6-62bf-4850-b9fd-30ac703445bc-proxy-tls\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:50:48.963016 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:48.962921    2551 generic.go:356] "Generic (PLEG): container finished" podID="0d2bfcc6-62bf-4850-b9fd-30ac703445bc" containerID="de0db827a3a0da63d9b7b627027ce559cd08612fa479847376ca8c10d7589ead" exitCode=0
Jul 01 14:50:48.963016 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:48.962968    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g"
Jul 01 14:50:48.963016 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:48.963004    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g" event={"ID":"0d2bfcc6-62bf-4850-b9fd-30ac703445bc","Type":"ContainerDied","Data":"de0db827a3a0da63d9b7b627027ce559cd08612fa479847376ca8c10d7589ead"}
Jul 01 14:50:48.963565 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:48.963077    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g" event={"ID":"0d2bfcc6-62bf-4850-b9fd-30ac703445bc","Type":"ContainerDied","Data":"54211983bfb1d14e2fff9a8e639dc994e8f2f4fcda8241caf727b2cee340a5b0"}
Jul 01 14:50:48.963565 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:48.963117    2551 scope.go:117] "RemoveContainer" containerID="83ce75fd7f77b60c0e05a9cc33ccffd8d62eebc62965ff3cc4e2b8b4c862705e"
Jul 01 14:50:48.963735 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:48.963664    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg" podUID="124c2e52-d9a7-48ce-86e5-0e24d7c801b1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Jul 01 14:50:48.972119 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:48.972102    2551 scope.go:117] "RemoveContainer" containerID="de0db827a3a0da63d9b7b627027ce559cd08612fa479847376ca8c10d7589ead"
Jul 01 14:50:48.979948 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:48.979931    2551 scope.go:117] "RemoveContainer" containerID="83ce75fd7f77b60c0e05a9cc33ccffd8d62eebc62965ff3cc4e2b8b4c862705e"
Jul 01 14:50:48.980216 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:50:48.980195    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83ce75fd7f77b60c0e05a9cc33ccffd8d62eebc62965ff3cc4e2b8b4c862705e\": container with ID starting with 83ce75fd7f77b60c0e05a9cc33ccffd8d62eebc62965ff3cc4e2b8b4c862705e not found: ID does not exist" containerID="83ce75fd7f77b60c0e05a9cc33ccffd8d62eebc62965ff3cc4e2b8b4c862705e"
Jul 01 14:50:48.980257 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:48.980226    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83ce75fd7f77b60c0e05a9cc33ccffd8d62eebc62965ff3cc4e2b8b4c862705e"} err="failed to get container status \"83ce75fd7f77b60c0e05a9cc33ccffd8d62eebc62965ff3cc4e2b8b4c862705e\": rpc error: code = NotFound desc = could not find container \"83ce75fd7f77b60c0e05a9cc33ccffd8d62eebc62965ff3cc4e2b8b4c862705e\": container with ID starting with 83ce75fd7f77b60c0e05a9cc33ccffd8d62eebc62965ff3cc4e2b8b4c862705e not found: ID does not exist"
Jul 01 14:50:48.980257 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:48.980243    2551 scope.go:117] "RemoveContainer" containerID="de0db827a3a0da63d9b7b627027ce559cd08612fa479847376ca8c10d7589ead"
Jul 01 14:50:48.980486 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:50:48.980468    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de0db827a3a0da63d9b7b627027ce559cd08612fa479847376ca8c10d7589ead\": container with ID starting with de0db827a3a0da63d9b7b627027ce559cd08612fa479847376ca8c10d7589ead not found: ID does not exist" containerID="de0db827a3a0da63d9b7b627027ce559cd08612fa479847376ca8c10d7589ead"
Jul 01 14:50:48.980522 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:48.980494    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de0db827a3a0da63d9b7b627027ce559cd08612fa479847376ca8c10d7589ead"} err="failed to get container status \"de0db827a3a0da63d9b7b627027ce559cd08612fa479847376ca8c10d7589ead\": rpc error: code = NotFound desc = could not find container \"de0db827a3a0da63d9b7b627027ce559cd08612fa479847376ca8c10d7589ead\": container with ID starting with de0db827a3a0da63d9b7b627027ce559cd08612fa479847376ca8c10d7589ead not found: ID does not exist"
Jul 01 14:50:48.984897 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:48.984874    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g"]
Jul 01 14:50:48.990979 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:48.990955    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g"]
Jul 01 14:50:49.103965 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:49.103933    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d2bfcc6-62bf-4850-b9fd-30ac703445bc" path="/var/lib/kubelet/pods/0d2bfcc6-62bf-4850-b9fd-30ac703445bc/volumes"
Jul 01 14:50:49.584156 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:49.584107    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-680c6-predictor-7d48c66ccf-x8h4g" podUID="0d2bfcc6-62bf-4850-b9fd-30ac703445bc" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.28:8643/healthz\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 01 14:50:50.099855 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:50.099814    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s"
Jul 01 14:50:53.968772 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:53.968742    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg"
Jul 01 14:50:53.969242 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:50:53.969213    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg" podUID="124c2e52-d9a7-48ce-86e5-0e24d7c801b1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Jul 01 14:51:03.969949 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:03.969905    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg" podUID="124c2e52-d9a7-48ce-86e5-0e24d7c801b1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Jul 01 14:51:13.970132 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:13.970044    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg" podUID="124c2e52-d9a7-48ce-86e5-0e24d7c801b1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Jul 01 14:51:23.969998 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:23.969959    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg" podUID="124c2e52-d9a7-48ce-86e5-0e24d7c801b1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Jul 01 14:51:33.970809 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:33.970776    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg"
Jul 01 14:51:34.973404 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:34.973364    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q"]
Jul 01 14:51:34.973962 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:34.973939    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d2bfcc6-62bf-4850-b9fd-30ac703445bc" containerName="kserve-container"
Jul 01 14:51:34.974016 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:34.973966    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d2bfcc6-62bf-4850-b9fd-30ac703445bc" containerName="kserve-container"
Jul 01 14:51:34.974016 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:34.973993    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d2bfcc6-62bf-4850-b9fd-30ac703445bc" containerName="kube-rbac-proxy"
Jul 01 14:51:34.974016 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:34.974001    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d2bfcc6-62bf-4850-b9fd-30ac703445bc" containerName="kube-rbac-proxy"
Jul 01 14:51:34.974105 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:34.974082    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d2bfcc6-62bf-4850-b9fd-30ac703445bc" containerName="kube-rbac-proxy"
Jul 01 14:51:34.974105 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:34.974098    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d2bfcc6-62bf-4850-b9fd-30ac703445bc" containerName="kserve-container"
Jul 01 14:51:34.977029 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:34.977005    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q"
Jul 01 14:51:34.979372 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:34.979347    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-0d08f-predictor-serving-cert\""
Jul 01 14:51:34.979590 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:34.979574    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-0d08f-kube-rbac-proxy-sar-config\""
Jul 01 14:51:34.988522 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:34.988497    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q"]
Jul 01 14:51:35.145101 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:35.145051    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b323f816-6b6d-4d84-bfe0-d3f00f776296-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-0d08f-predictor-7b6f897897-5bl7q\" (UID: \"b323f816-6b6d-4d84-bfe0-d3f00f776296\") " pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q"
Jul 01 14:51:35.145356 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:35.145114    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b323f816-6b6d-4d84-bfe0-d3f00f776296-proxy-tls\") pod \"success-200-isvc-0d08f-predictor-7b6f897897-5bl7q\" (UID: \"b323f816-6b6d-4d84-bfe0-d3f00f776296\") " pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q"
Jul 01 14:51:35.145356 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:35.145159    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gqrx\" (UniqueName: \"kubernetes.io/projected/b323f816-6b6d-4d84-bfe0-d3f00f776296-kube-api-access-4gqrx\") pod \"success-200-isvc-0d08f-predictor-7b6f897897-5bl7q\" (UID: \"b323f816-6b6d-4d84-bfe0-d3f00f776296\") " pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q"
Jul 01 14:51:35.214499 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:35.214464    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s"]
Jul 01 14:51:35.214807 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:35.214767    2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s" podUID="01046ee7-828b-4082-8f82-a9fe6bfa9ebc" containerName="kube-rbac-proxy" containerID="cri-o://6ed8b3d3f2852791b1e1ee96b7503c381fece3f84a207ed1eb9e56ded16202a0" gracePeriod=30
Jul 01 14:51:35.214913 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:35.214767    2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s" podUID="01046ee7-828b-4082-8f82-a9fe6bfa9ebc" containerName="kserve-container" containerID="cri-o://6e38051030a08853965de8939cfec3a35c3c14d2dbed41ce1aad2abc04a3830d" gracePeriod=30
Jul 01 14:51:35.246728 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:35.246688    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b323f816-6b6d-4d84-bfe0-d3f00f776296-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-0d08f-predictor-7b6f897897-5bl7q\" (UID: \"b323f816-6b6d-4d84-bfe0-d3f00f776296\") " pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q"
Jul 01 14:51:35.246925 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:35.246756    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b323f816-6b6d-4d84-bfe0-d3f00f776296-proxy-tls\") pod \"success-200-isvc-0d08f-predictor-7b6f897897-5bl7q\" (UID: \"b323f816-6b6d-4d84-bfe0-d3f00f776296\") " pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q"
Jul 01 14:51:35.246925 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:35.246792    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gqrx\" (UniqueName: \"kubernetes.io/projected/b323f816-6b6d-4d84-bfe0-d3f00f776296-kube-api-access-4gqrx\") pod \"success-200-isvc-0d08f-predictor-7b6f897897-5bl7q\" (UID: \"b323f816-6b6d-4d84-bfe0-d3f00f776296\") " pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q"
Jul 01 14:51:35.247049 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:51:35.246943    2551 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-serving-cert: secret "success-200-isvc-0d08f-predictor-serving-cert" not found
Jul 01 14:51:35.247049 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:51:35.247025    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b323f816-6b6d-4d84-bfe0-d3f00f776296-proxy-tls podName:b323f816-6b6d-4d84-bfe0-d3f00f776296 nodeName:}" failed. No retries permitted until 2026-07-01 14:51:35.747002068 +0000 UTC m=+733.075574365 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/b323f816-6b6d-4d84-bfe0-d3f00f776296-proxy-tls") pod "success-200-isvc-0d08f-predictor-7b6f897897-5bl7q" (UID: "b323f816-6b6d-4d84-bfe0-d3f00f776296") : secret "success-200-isvc-0d08f-predictor-serving-cert" not found
Jul 01 14:51:35.247383 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:35.247363    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b323f816-6b6d-4d84-bfe0-d3f00f776296-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-0d08f-predictor-7b6f897897-5bl7q\" (UID: \"b323f816-6b6d-4d84-bfe0-d3f00f776296\") " pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q"
Jul 01 14:51:35.256925 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:35.256887    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gqrx\" (UniqueName: \"kubernetes.io/projected/b323f816-6b6d-4d84-bfe0-d3f00f776296-kube-api-access-4gqrx\") pod \"success-200-isvc-0d08f-predictor-7b6f897897-5bl7q\" (UID: \"b323f816-6b6d-4d84-bfe0-d3f00f776296\") " pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q"
Jul 01 14:51:35.751280 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:35.751250    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b323f816-6b6d-4d84-bfe0-d3f00f776296-proxy-tls\") pod \"success-200-isvc-0d08f-predictor-7b6f897897-5bl7q\" (UID: \"b323f816-6b6d-4d84-bfe0-d3f00f776296\") " pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q"
Jul 01 14:51:35.753644 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:35.753604    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b323f816-6b6d-4d84-bfe0-d3f00f776296-proxy-tls\") pod \"success-200-isvc-0d08f-predictor-7b6f897897-5bl7q\" (UID: \"b323f816-6b6d-4d84-bfe0-d3f00f776296\") " pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q"
Jul 01 14:51:35.889292 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:35.889260    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q"
Jul 01 14:51:36.025576 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:36.025549    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q"]
Jul 01 14:51:36.027004 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:51:36.026980    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb323f816_6b6d_4d84_bfe0_d3f00f776296.slice/crio-74191a13d67dbbd2b53f7cae4f997c415af11ee6ede874235e4cdf44486f30bd WatchSource:0}: Error finding container 74191a13d67dbbd2b53f7cae4f997c415af11ee6ede874235e4cdf44486f30bd: Status 404 returned error can't find the container with id 74191a13d67dbbd2b53f7cae4f997c415af11ee6ede874235e4cdf44486f30bd
Jul 01 14:51:36.179375 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:36.179338    2551 generic.go:356] "Generic (PLEG): container finished" podID="01046ee7-828b-4082-8f82-a9fe6bfa9ebc" containerID="6ed8b3d3f2852791b1e1ee96b7503c381fece3f84a207ed1eb9e56ded16202a0" exitCode=2
Jul 01 14:51:36.179563 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:36.179421    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s" event={"ID":"01046ee7-828b-4082-8f82-a9fe6bfa9ebc","Type":"ContainerDied","Data":"6ed8b3d3f2852791b1e1ee96b7503c381fece3f84a207ed1eb9e56ded16202a0"}
Jul 01 14:51:36.181138 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:36.181112    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q" event={"ID":"b323f816-6b6d-4d84-bfe0-d3f00f776296","Type":"ContainerStarted","Data":"f049b2e41d4ab4fb1b45300ea1dab0b02f81d43612795915d8621cbe1535a5bd"}
Jul 01 14:51:36.181276 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:36.181167    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q" event={"ID":"b323f816-6b6d-4d84-bfe0-d3f00f776296","Type":"ContainerStarted","Data":"5b6e4cee605d6e500a260ba98ad5f588dcd0719f267316d6974b50a467384a49"}
Jul 01 14:51:36.181276 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:36.181199    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q" event={"ID":"b323f816-6b6d-4d84-bfe0-d3f00f776296","Type":"ContainerStarted","Data":"74191a13d67dbbd2b53f7cae4f997c415af11ee6ede874235e4cdf44486f30bd"}
Jul 01 14:51:36.181276 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:36.181232    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q"
Jul 01 14:51:36.181433 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:36.181349    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q"
Jul 01 14:51:36.182800 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:36.182776    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q" podUID="b323f816-6b6d-4d84-bfe0-d3f00f776296" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Jul 01 14:51:36.210009 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:36.209960    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q" podStartSLOduration=2.209939875 podStartE2EDuration="2.209939875s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:51:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:51:36.207790288 +0000 UTC m=+733.536362615" watchObservedRunningTime="2026-07-01 14:51:36.209939875 +0000 UTC m=+733.538512256"
Jul 01 14:51:36.624157 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:36.624110    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s" podUID="01046ee7-828b-4082-8f82-a9fe6bfa9ebc" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.29:8643/healthz\": dial tcp 10.133.0.29:8643: connect: connection refused"
Jul 01 14:51:37.186033 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:37.185992    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q" podUID="b323f816-6b6d-4d84-bfe0-d3f00f776296" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Jul 01 14:51:40.099164 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:40.099116    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s" podUID="01046ee7-828b-4082-8f82-a9fe6bfa9ebc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Jul 01 14:51:40.666346 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:40.666321    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s"
Jul 01 14:51:40.803439 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:40.803342    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-cabundle-cert\") pod \"01046ee7-828b-4082-8f82-a9fe6bfa9ebc\" (UID: \"01046ee7-828b-4082-8f82-a9fe6bfa9ebc\") "
Jul 01 14:51:40.803439 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:40.803391    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmfc5\" (UniqueName: \"kubernetes.io/projected/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-kube-api-access-mmfc5\") pod \"01046ee7-828b-4082-8f82-a9fe6bfa9ebc\" (UID: \"01046ee7-828b-4082-8f82-a9fe6bfa9ebc\") "
Jul 01 14:51:40.803794 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:40.803454    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-kserve-provision-location\") pod \"01046ee7-828b-4082-8f82-a9fe6bfa9ebc\" (UID: \"01046ee7-828b-4082-8f82-a9fe6bfa9ebc\") "
Jul 01 14:51:40.803794 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:40.803473    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-kube-rbac-proxy-sar-config\") pod \"01046ee7-828b-4082-8f82-a9fe6bfa9ebc\" (UID: \"01046ee7-828b-4082-8f82-a9fe6bfa9ebc\") "
Jul 01 14:51:40.803794 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:40.803493    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-proxy-tls\") pod \"01046ee7-828b-4082-8f82-a9fe6bfa9ebc\" (UID: \"01046ee7-828b-4082-8f82-a9fe6bfa9ebc\") "
Jul 01 14:51:40.803925 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:40.803830    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "01046ee7-828b-4082-8f82-a9fe6bfa9ebc" (UID: "01046ee7-828b-4082-8f82-a9fe6bfa9ebc"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:51:40.803925 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:40.803850    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "01046ee7-828b-4082-8f82-a9fe6bfa9ebc" (UID: "01046ee7-828b-4082-8f82-a9fe6bfa9ebc"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 14:51:40.803925 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:40.803834    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "01046ee7-828b-4082-8f82-a9fe6bfa9ebc" (UID: "01046ee7-828b-4082-8f82-a9fe6bfa9ebc"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 14:51:40.805587 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:40.805561    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "01046ee7-828b-4082-8f82-a9fe6bfa9ebc" (UID: "01046ee7-828b-4082-8f82-a9fe6bfa9ebc"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 14:51:40.805693 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:40.805632    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-kube-api-access-mmfc5" (OuterVolumeSpecName: "kube-api-access-mmfc5") pod "01046ee7-828b-4082-8f82-a9fe6bfa9ebc" (UID: "01046ee7-828b-4082-8f82-a9fe6bfa9ebc"). InnerVolumeSpecName "kube-api-access-mmfc5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 14:51:40.904971 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:40.904934    2551 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-kserve-provision-location\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:51:40.904971 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:40.904966    2551 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:51:40.904971 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:40.904976    2551 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-proxy-tls\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:51:40.905190 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:40.904987    2551 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-cabundle-cert\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:51:40.905190 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:40.904996    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmfc5\" (UniqueName: \"kubernetes.io/projected/01046ee7-828b-4082-8f82-a9fe6bfa9ebc-kube-api-access-mmfc5\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 14:51:41.206897 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:41.206866    2551 generic.go:356] "Generic (PLEG): container finished" podID="01046ee7-828b-4082-8f82-a9fe6bfa9ebc" containerID="6e38051030a08853965de8939cfec3a35c3c14d2dbed41ce1aad2abc04a3830d" exitCode=0
Jul 01 14:51:41.207283 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:41.206920    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s"
Jul 01 14:51:41.207283 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:41.206934    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s" event={"ID":"01046ee7-828b-4082-8f82-a9fe6bfa9ebc","Type":"ContainerDied","Data":"6e38051030a08853965de8939cfec3a35c3c14d2dbed41ce1aad2abc04a3830d"}
Jul 01 14:51:41.207283 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:41.206990    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s" event={"ID":"01046ee7-828b-4082-8f82-a9fe6bfa9ebc","Type":"ContainerDied","Data":"1b817432af365e796fa2b790781183ddc5fb5de0b2579dd3409ff24489d733e4"}
Jul 01 14:51:41.207283 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:41.207019    2551 scope.go:117] "RemoveContainer" containerID="6ed8b3d3f2852791b1e1ee96b7503c381fece3f84a207ed1eb9e56ded16202a0"
Jul 01 14:51:41.215077 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:41.215054    2551 scope.go:117] "RemoveContainer" containerID="6e38051030a08853965de8939cfec3a35c3c14d2dbed41ce1aad2abc04a3830d"
Jul 01 14:51:41.224473 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:41.224447    2551 scope.go:117] "RemoveContainer" containerID="231a64244f0ff9d223b6b4b5857ccba5fc218f7d609b178d573706dc5bbc8607"
Jul 01 14:51:41.228328 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:41.228301    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s"]
Jul 01 14:51:41.232325 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:41.232300    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-zkm5s"]
Jul 01 14:51:41.233350 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:41.233336    2551 scope.go:117] "RemoveContainer" containerID="6ed8b3d3f2852791b1e1ee96b7503c381fece3f84a207ed1eb9e56ded16202a0"
Jul 01 14:51:41.233683 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:51:41.233661    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ed8b3d3f2852791b1e1ee96b7503c381fece3f84a207ed1eb9e56ded16202a0\": container with ID starting with 6ed8b3d3f2852791b1e1ee96b7503c381fece3f84a207ed1eb9e56ded16202a0 not found: ID does not exist" containerID="6ed8b3d3f2852791b1e1ee96b7503c381fece3f84a207ed1eb9e56ded16202a0"
Jul 01 14:51:41.233791 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:41.233690    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ed8b3d3f2852791b1e1ee96b7503c381fece3f84a207ed1eb9e56ded16202a0"} err="failed to get container status \"6ed8b3d3f2852791b1e1ee96b7503c381fece3f84a207ed1eb9e56ded16202a0\": rpc error: code = NotFound desc = could not find container \"6ed8b3d3f2852791b1e1ee96b7503c381fece3f84a207ed1eb9e56ded16202a0\": container with ID starting with 6ed8b3d3f2852791b1e1ee96b7503c381fece3f84a207ed1eb9e56ded16202a0 not found: ID does not exist"
Jul 01 14:51:41.233791 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:41.233709    2551 scope.go:117] "RemoveContainer" containerID="6e38051030a08853965de8939cfec3a35c3c14d2dbed41ce1aad2abc04a3830d"
Jul 01 14:51:41.233953 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:51:41.233936    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e38051030a08853965de8939cfec3a35c3c14d2dbed41ce1aad2abc04a3830d\": container with ID starting with 6e38051030a08853965de8939cfec3a35c3c14d2dbed41ce1aad2abc04a3830d not found: ID does not exist" containerID="6e38051030a08853965de8939cfec3a35c3c14d2dbed41ce1aad2abc04a3830d"
Jul 01 14:51:41.233992 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:41.233957    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e38051030a08853965de8939cfec3a35c3c14d2dbed41ce1aad2abc04a3830d"} err="failed to get container status \"6e38051030a08853965de8939cfec3a35c3c14d2dbed41ce1aad2abc04a3830d\": rpc error: code = NotFound desc = could not find container \"6e38051030a08853965de8939cfec3a35c3c14d2dbed41ce1aad2abc04a3830d\": container with ID starting with 6e38051030a08853965de8939cfec3a35c3c14d2dbed41ce1aad2abc04a3830d not found: ID does not exist"
Jul 01 14:51:41.233992 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:41.233971    2551 scope.go:117] "RemoveContainer" containerID="231a64244f0ff9d223b6b4b5857ccba5fc218f7d609b178d573706dc5bbc8607"
Jul 01 14:51:41.234195 ip-10-0-139-55 kubenswrapper[2551]: E0701 14:51:41.234179    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"231a64244f0ff9d223b6b4b5857ccba5fc218f7d609b178d573706dc5bbc8607\": container with ID starting with 231a64244f0ff9d223b6b4b5857ccba5fc218f7d609b178d573706dc5bbc8607 not found: ID does not exist" containerID="231a64244f0ff9d223b6b4b5857ccba5fc218f7d609b178d573706dc5bbc8607"
Jul 01 14:51:41.234238 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:41.234202    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"231a64244f0ff9d223b6b4b5857ccba5fc218f7d609b178d573706dc5bbc8607"} err="failed to get container status \"231a64244f0ff9d223b6b4b5857ccba5fc218f7d609b178d573706dc5bbc8607\": rpc error: code = NotFound desc = could not find container \"231a64244f0ff9d223b6b4b5857ccba5fc218f7d609b178d573706dc5bbc8607\": container with ID starting with 231a64244f0ff9d223b6b4b5857ccba5fc218f7d609b178d573706dc5bbc8607 not found: ID does not exist"
Jul 01 14:51:42.190770 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:42.190735    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q"
Jul 01 14:51:42.191319 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:42.191290    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q" podUID="b323f816-6b6d-4d84-bfe0-d3f00f776296" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Jul 01 14:51:43.103888 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:43.103857    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01046ee7-828b-4082-8f82-a9fe6bfa9ebc" path="/var/lib/kubelet/pods/01046ee7-828b-4082-8f82-a9fe6bfa9ebc/volumes"
Jul 01 14:51:52.191337 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:51:52.191301    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q" podUID="b323f816-6b6d-4d84-bfe0-d3f00f776296" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Jul 01 14:52:02.191349 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:52:02.191306    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q" podUID="b323f816-6b6d-4d84-bfe0-d3f00f776296" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Jul 01 14:52:12.192250 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:52:12.192200    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q" podUID="b323f816-6b6d-4d84-bfe0-d3f00f776296" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Jul 01 14:52:22.191535 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:52:22.191491    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q" podUID="b323f816-6b6d-4d84-bfe0-d3f00f776296" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Jul 01 14:52:32.193207 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:52:32.193177    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q"
Jul 01 14:54:23.097126 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:54:23.097048    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/ovn-acl-logging/0.log"
Jul 01 14:54:23.101775 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:54:23.101744    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/ovn-acl-logging/0.log"
Jul 01 14:57:23.414274 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.414236    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56b95f574b-ghq9b"]
Jul 01 14:57:23.414868 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.414650    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01046ee7-828b-4082-8f82-a9fe6bfa9ebc" containerName="storage-initializer"
Jul 01 14:57:23.414868 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.414665    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="01046ee7-828b-4082-8f82-a9fe6bfa9ebc" containerName="storage-initializer"
Jul 01 14:57:23.414868 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.414674    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01046ee7-828b-4082-8f82-a9fe6bfa9ebc" containerName="kserve-container"
Jul 01 14:57:23.414868 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.414679    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="01046ee7-828b-4082-8f82-a9fe6bfa9ebc" containerName="kserve-container"
Jul 01 14:57:23.414868 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.414686    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01046ee7-828b-4082-8f82-a9fe6bfa9ebc" containerName="kube-rbac-proxy"
Jul 01 14:57:23.414868 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.414692    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="01046ee7-828b-4082-8f82-a9fe6bfa9ebc" containerName="kube-rbac-proxy"
Jul 01 14:57:23.414868 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.414761    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="01046ee7-828b-4082-8f82-a9fe6bfa9ebc" containerName="kube-rbac-proxy"
Jul 01 14:57:23.414868 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.414770    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="01046ee7-828b-4082-8f82-a9fe6bfa9ebc" containerName="kserve-container"
Jul 01 14:57:23.416753 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.416733    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56b95f574b-ghq9b"
Jul 01 14:57:23.419578 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.419558    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 01 14:57:23.420343 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.420320    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 01 14:57:23.420343 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.420337    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 01 14:57:23.420514 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.420365    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 01 14:57:23.420514 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.420339    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 01 14:57:23.420514 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.420323    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-x7qb4\""
Jul 01 14:57:23.425912 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.425893    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 01 14:57:23.436686 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.436663    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56b95f574b-ghq9b"]
Jul 01 14:57:23.488329 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.488286    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjkzx\" (UniqueName: \"kubernetes.io/projected/3499eb6d-ae9e-4448-af49-901e8091f9f3-kube-api-access-hjkzx\") pod \"console-56b95f574b-ghq9b\" (UID: \"3499eb6d-ae9e-4448-af49-901e8091f9f3\") " pod="openshift-console/console-56b95f574b-ghq9b"
Jul 01 14:57:23.488329 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.488334    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3499eb6d-ae9e-4448-af49-901e8091f9f3-console-serving-cert\") pod \"console-56b95f574b-ghq9b\" (UID: \"3499eb6d-ae9e-4448-af49-901e8091f9f3\") " pod="openshift-console/console-56b95f574b-ghq9b"
Jul 01 14:57:23.488555 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.488388    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3499eb6d-ae9e-4448-af49-901e8091f9f3-console-oauth-config\") pod \"console-56b95f574b-ghq9b\" (UID: \"3499eb6d-ae9e-4448-af49-901e8091f9f3\") " pod="openshift-console/console-56b95f574b-ghq9b"
Jul 01 14:57:23.488555 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.488407    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3499eb6d-ae9e-4448-af49-901e8091f9f3-console-config\") pod \"console-56b95f574b-ghq9b\" (UID: \"3499eb6d-ae9e-4448-af49-901e8091f9f3\") " pod="openshift-console/console-56b95f574b-ghq9b"
Jul 01 14:57:23.488555 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.488430    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3499eb6d-ae9e-4448-af49-901e8091f9f3-oauth-serving-cert\") pod \"console-56b95f574b-ghq9b\" (UID: \"3499eb6d-ae9e-4448-af49-901e8091f9f3\") " pod="openshift-console/console-56b95f574b-ghq9b"
Jul 01 14:57:23.488555 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.488486    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3499eb6d-ae9e-4448-af49-901e8091f9f3-service-ca\") pod \"console-56b95f574b-ghq9b\" (UID: \"3499eb6d-ae9e-4448-af49-901e8091f9f3\") " pod="openshift-console/console-56b95f574b-ghq9b"
Jul 01 14:57:23.488555 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.488522    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3499eb6d-ae9e-4448-af49-901e8091f9f3-trusted-ca-bundle\") pod \"console-56b95f574b-ghq9b\" (UID: \"3499eb6d-ae9e-4448-af49-901e8091f9f3\") " pod="openshift-console/console-56b95f574b-ghq9b"
Jul 01 14:57:23.589448 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.589409    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjkzx\" (UniqueName: \"kubernetes.io/projected/3499eb6d-ae9e-4448-af49-901e8091f9f3-kube-api-access-hjkzx\") pod \"console-56b95f574b-ghq9b\" (UID: \"3499eb6d-ae9e-4448-af49-901e8091f9f3\") " pod="openshift-console/console-56b95f574b-ghq9b"
Jul 01 14:57:23.589448 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.589455    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3499eb6d-ae9e-4448-af49-901e8091f9f3-console-serving-cert\") pod \"console-56b95f574b-ghq9b\" (UID: \"3499eb6d-ae9e-4448-af49-901e8091f9f3\") " pod="openshift-console/console-56b95f574b-ghq9b"
Jul 01 14:57:23.589725 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.589489    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3499eb6d-ae9e-4448-af49-901e8091f9f3-console-oauth-config\") pod \"console-56b95f574b-ghq9b\" (UID: \"3499eb6d-ae9e-4448-af49-901e8091f9f3\") " pod="openshift-console/console-56b95f574b-ghq9b"
Jul 01 14:57:23.589725 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.589517    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3499eb6d-ae9e-4448-af49-901e8091f9f3-console-config\") pod \"console-56b95f574b-ghq9b\" (UID: \"3499eb6d-ae9e-4448-af49-901e8091f9f3\") " pod="openshift-console/console-56b95f574b-ghq9b"
Jul 01 14:57:23.589725 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.589538    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3499eb6d-ae9e-4448-af49-901e8091f9f3-oauth-serving-cert\") pod \"console-56b95f574b-ghq9b\" (UID: \"3499eb6d-ae9e-4448-af49-901e8091f9f3\") " pod="openshift-console/console-56b95f574b-ghq9b"
Jul 01 14:57:23.589725 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.589571    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3499eb6d-ae9e-4448-af49-901e8091f9f3-service-ca\") pod \"console-56b95f574b-ghq9b\" (UID: \"3499eb6d-ae9e-4448-af49-901e8091f9f3\") " pod="openshift-console/console-56b95f574b-ghq9b"
Jul 01 14:57:23.589725 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.589599    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3499eb6d-ae9e-4448-af49-901e8091f9f3-trusted-ca-bundle\") pod \"console-56b95f574b-ghq9b\" (UID: \"3499eb6d-ae9e-4448-af49-901e8091f9f3\") " pod="openshift-console/console-56b95f574b-ghq9b"
Jul 01 14:57:23.590370 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.590344    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3499eb6d-ae9e-4448-af49-901e8091f9f3-console-config\") pod \"console-56b95f574b-ghq9b\" (UID: \"3499eb6d-ae9e-4448-af49-901e8091f9f3\") " pod="openshift-console/console-56b95f574b-ghq9b"
Jul 01 14:57:23.590480 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.590463    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3499eb6d-ae9e-4448-af49-901e8091f9f3-service-ca\") pod \"console-56b95f574b-ghq9b\" (UID: \"3499eb6d-ae9e-4448-af49-901e8091f9f3\") " pod="openshift-console/console-56b95f574b-ghq9b"
Jul 01 14:57:23.590521 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.590505    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3499eb6d-ae9e-4448-af49-901e8091f9f3-oauth-serving-cert\") pod \"console-56b95f574b-ghq9b\" (UID: \"3499eb6d-ae9e-4448-af49-901e8091f9f3\") " pod="openshift-console/console-56b95f574b-ghq9b"
Jul 01 14:57:23.590588 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.590566    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3499eb6d-ae9e-4448-af49-901e8091f9f3-trusted-ca-bundle\") pod \"console-56b95f574b-ghq9b\" (UID: \"3499eb6d-ae9e-4448-af49-901e8091f9f3\") " pod="openshift-console/console-56b95f574b-ghq9b"
Jul 01 14:57:23.592186 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.592158    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3499eb6d-ae9e-4448-af49-901e8091f9f3-console-oauth-config\") pod \"console-56b95f574b-ghq9b\" (UID: \"3499eb6d-ae9e-4448-af49-901e8091f9f3\") " pod="openshift-console/console-56b95f574b-ghq9b"
Jul 01 14:57:23.592257 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.592192    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3499eb6d-ae9e-4448-af49-901e8091f9f3-console-serving-cert\") pod \"console-56b95f574b-ghq9b\" (UID: \"3499eb6d-ae9e-4448-af49-901e8091f9f3\") " pod="openshift-console/console-56b95f574b-ghq9b"
Jul 01 14:57:23.600464 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.600438    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjkzx\" (UniqueName: \"kubernetes.io/projected/3499eb6d-ae9e-4448-af49-901e8091f9f3-kube-api-access-hjkzx\") pod \"console-56b95f574b-ghq9b\" (UID: \"3499eb6d-ae9e-4448-af49-901e8091f9f3\") " pod="openshift-console/console-56b95f574b-ghq9b"
Jul 01 14:57:23.727072 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.727040    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56b95f574b-ghq9b"
Jul 01 14:57:23.864423 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:23.864357    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56b95f574b-ghq9b"]
Jul 01 14:57:23.867318 ip-10-0-139-55 kubenswrapper[2551]: W0701 14:57:23.867283    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3499eb6d_ae9e_4448_af49_901e8091f9f3.slice/crio-58b5abe098aca2db70cbc9e9e5360566bf378729cb470006d606967cea0fafac WatchSource:0}: Error finding container 58b5abe098aca2db70cbc9e9e5360566bf378729cb470006d606967cea0fafac: Status 404 returned error can't find the container with id 58b5abe098aca2db70cbc9e9e5360566bf378729cb470006d606967cea0fafac
Jul 01 14:57:24.669212 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:24.669177    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56b95f574b-ghq9b" event={"ID":"3499eb6d-ae9e-4448-af49-901e8091f9f3","Type":"ContainerStarted","Data":"c8a2c309826f9c2d1b3bb783830c2b8f07b948c128fd840084e9f1e134749b04"}
Jul 01 14:57:24.669611 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:24.669231    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56b95f574b-ghq9b" event={"ID":"3499eb6d-ae9e-4448-af49-901e8091f9f3","Type":"ContainerStarted","Data":"58b5abe098aca2db70cbc9e9e5360566bf378729cb470006d606967cea0fafac"}
Jul 01 14:57:24.686840 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:24.686776    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56b95f574b-ghq9b" podStartSLOduration=1.686753891 podStartE2EDuration="1.686753891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:57:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 14:57:24.68657998 +0000 UTC m=+1082.015152397" watchObservedRunningTime="2026-07-01 14:57:24.686753891 +0000 UTC m=+1082.015326199"
Jul 01 14:57:33.727375 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:33.727344    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56b95f574b-ghq9b"
Jul 01 14:57:33.727756 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:33.727405    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56b95f574b-ghq9b"
Jul 01 14:57:33.732278 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:33.732252    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56b95f574b-ghq9b"
Jul 01 14:57:34.715759 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:57:34.715729    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56b95f574b-ghq9b"
Jul 01 14:59:23.127397 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:59:23.127320    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/ovn-acl-logging/0.log"
Jul 01 14:59:23.130159 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:59:23.130137    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/ovn-acl-logging/0.log"
Jul 01 14:59:59.966972 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:59:59.966932    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg"]
Jul 01 14:59:59.967825 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:59:59.967769    2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg" podUID="124c2e52-d9a7-48ce-86e5-0e24d7c801b1" containerName="kserve-container" containerID="cri-o://925b52940735f4fd596c2dc655b97cee159cdc820c7b15b3714c1f200c80e4a6" gracePeriod=30
Jul 01 14:59:59.968219 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:59:59.968058    2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg" podUID="124c2e52-d9a7-48ce-86e5-0e24d7c801b1" containerName="kube-rbac-proxy" containerID="cri-o://e624e0a11ae75c90c83b6d0dc6043ecb872d8bf16f8f595cab6ee61c29e0f408" gracePeriod=30
Jul 01 14:59:59.969417 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:59:59.969394    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7"]
Jul 01 14:59:59.973665 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:59:59.973113    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7"
Jul 01 14:59:59.975451 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:59:59.975431    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-7bec4-kube-rbac-proxy-sar-config\""
Jul 01 14:59:59.975795 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:59:59.975772    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-7bec4-predictor-serving-cert\""
Jul 01 14:59:59.981518 ip-10-0-139-55 kubenswrapper[2551]: I0701 14:59:59.981499    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7"]
Jul 01 15:00:00.044936 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:00.044902    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbggq\" (UniqueName: \"kubernetes.io/projected/be6143cc-5b12-440c-9a8b-d562b6f874bf-kube-api-access-xbggq\") pod \"success-200-isvc-7bec4-predictor-7d98f4b667-58sb7\" (UID: \"be6143cc-5b12-440c-9a8b-d562b6f874bf\") " pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7"
Jul 01 15:00:00.045124 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:00.044950    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/be6143cc-5b12-440c-9a8b-d562b6f874bf-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-7bec4-predictor-7d98f4b667-58sb7\" (UID: \"be6143cc-5b12-440c-9a8b-d562b6f874bf\") " pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7"
Jul 01 15:00:00.045124 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:00.045032    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be6143cc-5b12-440c-9a8b-d562b6f874bf-proxy-tls\") pod \"success-200-isvc-7bec4-predictor-7d98f4b667-58sb7\" (UID: \"be6143cc-5b12-440c-9a8b-d562b6f874bf\") " pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7"
Jul 01 15:00:00.145932 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:00.145888    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/be6143cc-5b12-440c-9a8b-d562b6f874bf-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-7bec4-predictor-7d98f4b667-58sb7\" (UID: \"be6143cc-5b12-440c-9a8b-d562b6f874bf\") " pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7"
Jul 01 15:00:00.146133 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:00.145958    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be6143cc-5b12-440c-9a8b-d562b6f874bf-proxy-tls\") pod \"success-200-isvc-7bec4-predictor-7d98f4b667-58sb7\" (UID: \"be6143cc-5b12-440c-9a8b-d562b6f874bf\") " pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7"
Jul 01 15:00:00.146133 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:00.146014    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbggq\" (UniqueName: \"kubernetes.io/projected/be6143cc-5b12-440c-9a8b-d562b6f874bf-kube-api-access-xbggq\") pod \"success-200-isvc-7bec4-predictor-7d98f4b667-58sb7\" (UID: \"be6143cc-5b12-440c-9a8b-d562b6f874bf\") " pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7"
Jul 01 15:00:00.146133 ip-10-0-139-55 kubenswrapper[2551]: E0701 15:00:00.146109    2551 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-serving-cert: secret "success-200-isvc-7bec4-predictor-serving-cert" not found
Jul 01 15:00:00.146302 ip-10-0-139-55 kubenswrapper[2551]: E0701 15:00:00.146195    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be6143cc-5b12-440c-9a8b-d562b6f874bf-proxy-tls podName:be6143cc-5b12-440c-9a8b-d562b6f874bf nodeName:}" failed. No retries permitted until 2026-07-01 15:00:00.646175775 +0000 UTC m=+1237.974748074 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/be6143cc-5b12-440c-9a8b-d562b6f874bf-proxy-tls") pod "success-200-isvc-7bec4-predictor-7d98f4b667-58sb7" (UID: "be6143cc-5b12-440c-9a8b-d562b6f874bf") : secret "success-200-isvc-7bec4-predictor-serving-cert" not found
Jul 01 15:00:00.146550 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:00.146529    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/be6143cc-5b12-440c-9a8b-d562b6f874bf-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-7bec4-predictor-7d98f4b667-58sb7\" (UID: \"be6143cc-5b12-440c-9a8b-d562b6f874bf\") " pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7"
Jul 01 15:00:00.155717 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:00.155691    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbggq\" (UniqueName: \"kubernetes.io/projected/be6143cc-5b12-440c-9a8b-d562b6f874bf-kube-api-access-xbggq\") pod \"success-200-isvc-7bec4-predictor-7d98f4b667-58sb7\" (UID: \"be6143cc-5b12-440c-9a8b-d562b6f874bf\") " pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7"
Jul 01 15:00:00.359615 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:00.359581    2551 generic.go:356] "Generic (PLEG): container finished" podID="124c2e52-d9a7-48ce-86e5-0e24d7c801b1" containerID="e624e0a11ae75c90c83b6d0dc6043ecb872d8bf16f8f595cab6ee61c29e0f408" exitCode=2
Jul 01 15:00:00.359815 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:00.359648    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg" event={"ID":"124c2e52-d9a7-48ce-86e5-0e24d7c801b1","Type":"ContainerDied","Data":"e624e0a11ae75c90c83b6d0dc6043ecb872d8bf16f8f595cab6ee61c29e0f408"}
Jul 01 15:00:00.650975 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:00.650863    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be6143cc-5b12-440c-9a8b-d562b6f874bf-proxy-tls\") pod \"success-200-isvc-7bec4-predictor-7d98f4b667-58sb7\" (UID: \"be6143cc-5b12-440c-9a8b-d562b6f874bf\") " pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7"
Jul 01 15:00:00.653293 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:00.653274    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be6143cc-5b12-440c-9a8b-d562b6f874bf-proxy-tls\") pod \"success-200-isvc-7bec4-predictor-7d98f4b667-58sb7\" (UID: \"be6143cc-5b12-440c-9a8b-d562b6f874bf\") " pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7"
Jul 01 15:00:00.892140 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:00.892111    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7"
Jul 01 15:00:01.028007 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:01.027981    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7"]
Jul 01 15:00:01.030532 ip-10-0-139-55 kubenswrapper[2551]: W0701 15:00:01.030502    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe6143cc_5b12_440c_9a8b_d562b6f874bf.slice/crio-2d93469799a72ed73f0b884654cc3ecb78d18523ca7dd5d8016b597a6ba90ba4 WatchSource:0}: Error finding container 2d93469799a72ed73f0b884654cc3ecb78d18523ca7dd5d8016b597a6ba90ba4: Status 404 returned error can't find the container with id 2d93469799a72ed73f0b884654cc3ecb78d18523ca7dd5d8016b597a6ba90ba4
Jul 01 15:00:01.366673 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:01.366606    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7" event={"ID":"be6143cc-5b12-440c-9a8b-d562b6f874bf","Type":"ContainerStarted","Data":"da14dbfa2450f7086763563fef893fb1f094592883f219c0bb67a6369dfb1722"}
Jul 01 15:00:01.366885 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:01.366695    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7" event={"ID":"be6143cc-5b12-440c-9a8b-d562b6f874bf","Type":"ContainerStarted","Data":"c3c9781939becb336d133baa6cbe5ebff46624224304ab8db2e1b748fdf467e9"}
Jul 01 15:00:01.366885 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:01.366722    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7"
Jul 01 15:00:01.366885 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:01.366828    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7"
Jul 01 15:00:01.366885 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:01.366879    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7" event={"ID":"be6143cc-5b12-440c-9a8b-d562b6f874bf","Type":"ContainerStarted","Data":"2d93469799a72ed73f0b884654cc3ecb78d18523ca7dd5d8016b597a6ba90ba4"}
Jul 01 15:00:01.368409 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:01.368384    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7" podUID="be6143cc-5b12-440c-9a8b-d562b6f874bf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 01 15:00:01.388685 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:01.387764    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7" podStartSLOduration=2.387741453 podStartE2EDuration="2.387741453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 14:59:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 15:00:01.384681131 +0000 UTC m=+1238.713253438" watchObservedRunningTime="2026-07-01 15:00:01.387741453 +0000 UTC m=+1238.716313759"
Jul 01 15:00:02.370736 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:02.370691    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7" podUID="be6143cc-5b12-440c-9a8b-d562b6f874bf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 01 15:00:03.964236 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:03.964195    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg" podUID="124c2e52-d9a7-48ce-86e5-0e24d7c801b1" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.30:8643/healthz\": dial tcp 10.133.0.30:8643: connect: connection refused"
Jul 01 15:00:03.969665 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:03.969598    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg" podUID="124c2e52-d9a7-48ce-86e5-0e24d7c801b1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Jul 01 15:00:07.026252 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:07.026218    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg"
Jul 01 15:00:07.105793 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:07.105704    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/124c2e52-d9a7-48ce-86e5-0e24d7c801b1-kube-rbac-proxy-sar-config\") pod \"124c2e52-d9a7-48ce-86e5-0e24d7c801b1\" (UID: \"124c2e52-d9a7-48ce-86e5-0e24d7c801b1\") "
Jul 01 15:00:07.105793 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:07.105738    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jx65b\" (UniqueName: \"kubernetes.io/projected/124c2e52-d9a7-48ce-86e5-0e24d7c801b1-kube-api-access-jx65b\") pod \"124c2e52-d9a7-48ce-86e5-0e24d7c801b1\" (UID: \"124c2e52-d9a7-48ce-86e5-0e24d7c801b1\") "
Jul 01 15:00:07.105793 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:07.105767    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/124c2e52-d9a7-48ce-86e5-0e24d7c801b1-proxy-tls\") pod \"124c2e52-d9a7-48ce-86e5-0e24d7c801b1\" (UID: \"124c2e52-d9a7-48ce-86e5-0e24d7c801b1\") "
Jul 01 15:00:07.106135 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:07.106104    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/124c2e52-d9a7-48ce-86e5-0e24d7c801b1-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "124c2e52-d9a7-48ce-86e5-0e24d7c801b1" (UID: "124c2e52-d9a7-48ce-86e5-0e24d7c801b1"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 15:00:07.107892 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:07.107860    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/124c2e52-d9a7-48ce-86e5-0e24d7c801b1-kube-api-access-jx65b" (OuterVolumeSpecName: "kube-api-access-jx65b") pod "124c2e52-d9a7-48ce-86e5-0e24d7c801b1" (UID: "124c2e52-d9a7-48ce-86e5-0e24d7c801b1"). InnerVolumeSpecName "kube-api-access-jx65b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 15:00:07.107963 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:07.107886    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/124c2e52-d9a7-48ce-86e5-0e24d7c801b1-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "124c2e52-d9a7-48ce-86e5-0e24d7c801b1" (UID: "124c2e52-d9a7-48ce-86e5-0e24d7c801b1"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 15:00:07.207106 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:07.207067    2551 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/124c2e52-d9a7-48ce-86e5-0e24d7c801b1-proxy-tls\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 15:00:07.207106 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:07.207098    2551 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/124c2e52-d9a7-48ce-86e5-0e24d7c801b1-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 15:00:07.207106 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:07.207110    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jx65b\" (UniqueName: \"kubernetes.io/projected/124c2e52-d9a7-48ce-86e5-0e24d7c801b1-kube-api-access-jx65b\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 15:00:07.375951 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:07.375870    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7"
Jul 01 15:00:07.376496 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:07.376469    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7" podUID="be6143cc-5b12-440c-9a8b-d562b6f874bf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 01 15:00:07.393197 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:07.393156    2551 generic.go:356] "Generic (PLEG): container finished" podID="124c2e52-d9a7-48ce-86e5-0e24d7c801b1" containerID="925b52940735f4fd596c2dc655b97cee159cdc820c7b15b3714c1f200c80e4a6" exitCode=0
Jul 01 15:00:07.393414 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:07.393223    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg"
Jul 01 15:00:07.393414 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:07.393247    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg" event={"ID":"124c2e52-d9a7-48ce-86e5-0e24d7c801b1","Type":"ContainerDied","Data":"925b52940735f4fd596c2dc655b97cee159cdc820c7b15b3714c1f200c80e4a6"}
Jul 01 15:00:07.393414 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:07.393327    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg" event={"ID":"124c2e52-d9a7-48ce-86e5-0e24d7c801b1","Type":"ContainerDied","Data":"c96cc05553633b4bf8f09e9d4819f75e8be435e4aa8722855df3e25d76ff27b5"}
Jul 01 15:00:07.393414 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:07.393365    2551 scope.go:117] "RemoveContainer" containerID="e624e0a11ae75c90c83b6d0dc6043ecb872d8bf16f8f595cab6ee61c29e0f408"
Jul 01 15:00:07.403082 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:07.403059    2551 scope.go:117] "RemoveContainer" containerID="925b52940735f4fd596c2dc655b97cee159cdc820c7b15b3714c1f200c80e4a6"
Jul 01 15:00:07.411226 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:07.411201    2551 scope.go:117] "RemoveContainer" containerID="e624e0a11ae75c90c83b6d0dc6043ecb872d8bf16f8f595cab6ee61c29e0f408"
Jul 01 15:00:07.411529 ip-10-0-139-55 kubenswrapper[2551]: E0701 15:00:07.411510    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e624e0a11ae75c90c83b6d0dc6043ecb872d8bf16f8f595cab6ee61c29e0f408\": container with ID starting with e624e0a11ae75c90c83b6d0dc6043ecb872d8bf16f8f595cab6ee61c29e0f408 not found: ID does not exist" containerID="e624e0a11ae75c90c83b6d0dc6043ecb872d8bf16f8f595cab6ee61c29e0f408"
Jul 01 15:00:07.411585 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:07.411539    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e624e0a11ae75c90c83b6d0dc6043ecb872d8bf16f8f595cab6ee61c29e0f408"} err="failed to get container status \"e624e0a11ae75c90c83b6d0dc6043ecb872d8bf16f8f595cab6ee61c29e0f408\": rpc error: code = NotFound desc = could not find container \"e624e0a11ae75c90c83b6d0dc6043ecb872d8bf16f8f595cab6ee61c29e0f408\": container with ID starting with e624e0a11ae75c90c83b6d0dc6043ecb872d8bf16f8f595cab6ee61c29e0f408 not found: ID does not exist"
Jul 01 15:00:07.411585 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:07.411560    2551 scope.go:117] "RemoveContainer" containerID="925b52940735f4fd596c2dc655b97cee159cdc820c7b15b3714c1f200c80e4a6"
Jul 01 15:00:07.411804 ip-10-0-139-55 kubenswrapper[2551]: E0701 15:00:07.411786    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"925b52940735f4fd596c2dc655b97cee159cdc820c7b15b3714c1f200c80e4a6\": container with ID starting with 925b52940735f4fd596c2dc655b97cee159cdc820c7b15b3714c1f200c80e4a6 not found: ID does not exist" containerID="925b52940735f4fd596c2dc655b97cee159cdc820c7b15b3714c1f200c80e4a6"
Jul 01 15:00:07.411848 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:07.411809    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"925b52940735f4fd596c2dc655b97cee159cdc820c7b15b3714c1f200c80e4a6"} err="failed to get container status \"925b52940735f4fd596c2dc655b97cee159cdc820c7b15b3714c1f200c80e4a6\": rpc error: code = NotFound desc = could not find container \"925b52940735f4fd596c2dc655b97cee159cdc820c7b15b3714c1f200c80e4a6\": container with ID starting with 925b52940735f4fd596c2dc655b97cee159cdc820c7b15b3714c1f200c80e4a6 not found: ID does not exist"
Jul 01 15:00:07.414273 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:07.414246    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg"]
Jul 01 15:00:07.417975 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:07.417953    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-3297d-predictor-5494fbf467-jmcgg"]
Jul 01 15:00:09.103373 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:09.103334    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="124c2e52-d9a7-48ce-86e5-0e24d7c801b1" path="/var/lib/kubelet/pods/124c2e52-d9a7-48ce-86e5-0e24d7c801b1/volumes"
Jul 01 15:00:17.376490 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:17.376399    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7" podUID="be6143cc-5b12-440c-9a8b-d562b6f874bf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 01 15:00:27.377300 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:27.377258    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7" podUID="be6143cc-5b12-440c-9a8b-d562b6f874bf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 01 15:00:37.376510 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:37.376467    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7" podUID="be6143cc-5b12-440c-9a8b-d562b6f874bf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 01 15:00:47.376839 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:47.376791    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7" podUID="be6143cc-5b12-440c-9a8b-d562b6f874bf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 01 15:00:57.377799 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:57.377769    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7"
Jul 01 15:00:59.776957 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:59.776922    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q"]
Jul 01 15:00:59.777370 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:59.777179    2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q" podUID="b323f816-6b6d-4d84-bfe0-d3f00f776296" containerName="kserve-container" containerID="cri-o://5b6e4cee605d6e500a260ba98ad5f588dcd0719f267316d6974b50a467384a49" gracePeriod=30
Jul 01 15:00:59.777370 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:59.777261    2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q" podUID="b323f816-6b6d-4d84-bfe0-d3f00f776296" containerName="kube-rbac-proxy" containerID="cri-o://f049b2e41d4ab4fb1b45300ea1dab0b02f81d43612795915d8621cbe1535a5bd" gracePeriod=30
Jul 01 15:00:59.807039 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:59.807003    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq"]
Jul 01 15:00:59.807370 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:59.807356    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="124c2e52-d9a7-48ce-86e5-0e24d7c801b1" containerName="kube-rbac-proxy"
Jul 01 15:00:59.807417 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:59.807371    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="124c2e52-d9a7-48ce-86e5-0e24d7c801b1" containerName="kube-rbac-proxy"
Jul 01 15:00:59.807417 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:59.807389    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="124c2e52-d9a7-48ce-86e5-0e24d7c801b1" containerName="kserve-container"
Jul 01 15:00:59.807417 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:59.807394    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="124c2e52-d9a7-48ce-86e5-0e24d7c801b1" containerName="kserve-container"
Jul 01 15:00:59.807511 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:59.807448    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="124c2e52-d9a7-48ce-86e5-0e24d7c801b1" containerName="kserve-container"
Jul 01 15:00:59.807511 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:59.807456    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="124c2e52-d9a7-48ce-86e5-0e24d7c801b1" containerName="kube-rbac-proxy"
Jul 01 15:00:59.810597 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:59.810574    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq"
Jul 01 15:00:59.813050 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:59.813027    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-102b8-kube-rbac-proxy-sar-config\""
Jul 01 15:00:59.813178 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:59.813061    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-102b8-predictor-serving-cert\""
Jul 01 15:00:59.821607 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:59.821580    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq"]
Jul 01 15:00:59.861266 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:59.861234    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a136e7df-7ce1-434c-a08b-a08a8fc6bba4-proxy-tls\") pod \"success-200-isvc-102b8-predictor-784fbddbc5-gqzqq\" (UID: \"a136e7df-7ce1-434c-a08b-a08a8fc6bba4\") " pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq"
Jul 01 15:00:59.861431 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:59.861296    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a136e7df-7ce1-434c-a08b-a08a8fc6bba4-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-102b8-predictor-784fbddbc5-gqzqq\" (UID: \"a136e7df-7ce1-434c-a08b-a08a8fc6bba4\") " pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq"
Jul 01 15:00:59.861431 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:59.861341    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frg6r\" (UniqueName: \"kubernetes.io/projected/a136e7df-7ce1-434c-a08b-a08a8fc6bba4-kube-api-access-frg6r\") pod \"success-200-isvc-102b8-predictor-784fbddbc5-gqzqq\" (UID: \"a136e7df-7ce1-434c-a08b-a08a8fc6bba4\") " pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq"
Jul 01 15:00:59.962291 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:59.962255    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a136e7df-7ce1-434c-a08b-a08a8fc6bba4-proxy-tls\") pod \"success-200-isvc-102b8-predictor-784fbddbc5-gqzqq\" (UID: \"a136e7df-7ce1-434c-a08b-a08a8fc6bba4\") " pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq"
Jul 01 15:00:59.962470 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:59.962322    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a136e7df-7ce1-434c-a08b-a08a8fc6bba4-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-102b8-predictor-784fbddbc5-gqzqq\" (UID: \"a136e7df-7ce1-434c-a08b-a08a8fc6bba4\") " pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq"
Jul 01 15:00:59.962470 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:59.962386    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frg6r\" (UniqueName: \"kubernetes.io/projected/a136e7df-7ce1-434c-a08b-a08a8fc6bba4-kube-api-access-frg6r\") pod \"success-200-isvc-102b8-predictor-784fbddbc5-gqzqq\" (UID: \"a136e7df-7ce1-434c-a08b-a08a8fc6bba4\") " pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq"
Jul 01 15:00:59.963131 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:59.963113    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a136e7df-7ce1-434c-a08b-a08a8fc6bba4-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-102b8-predictor-784fbddbc5-gqzqq\" (UID: \"a136e7df-7ce1-434c-a08b-a08a8fc6bba4\") " pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq"
Jul 01 15:00:59.964874 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:59.964850    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a136e7df-7ce1-434c-a08b-a08a8fc6bba4-proxy-tls\") pod \"success-200-isvc-102b8-predictor-784fbddbc5-gqzqq\" (UID: \"a136e7df-7ce1-434c-a08b-a08a8fc6bba4\") " pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq"
Jul 01 15:00:59.973434 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:00:59.973405    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frg6r\" (UniqueName: \"kubernetes.io/projected/a136e7df-7ce1-434c-a08b-a08a8fc6bba4-kube-api-access-frg6r\") pod \"success-200-isvc-102b8-predictor-784fbddbc5-gqzqq\" (UID: \"a136e7df-7ce1-434c-a08b-a08a8fc6bba4\") " pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq"
Jul 01 15:01:00.122884 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:00.122790    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq"
Jul 01 15:01:00.266272 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:00.266243    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq"]
Jul 01 15:01:00.268152 ip-10-0-139-55 kubenswrapper[2551]: W0701 15:01:00.268124    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda136e7df_7ce1_434c_a08b_a08a8fc6bba4.slice/crio-39976bf7df710749af0f7708adaca7cd588e33b97ffa47c285bdaa1a3524db36 WatchSource:0}: Error finding container 39976bf7df710749af0f7708adaca7cd588e33b97ffa47c285bdaa1a3524db36: Status 404 returned error can't find the container with id 39976bf7df710749af0f7708adaca7cd588e33b97ffa47c285bdaa1a3524db36
Jul 01 15:01:00.619267 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:00.619229    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq" event={"ID":"a136e7df-7ce1-434c-a08b-a08a8fc6bba4","Type":"ContainerStarted","Data":"c6651d39789a2096dcaf426a0aa8fbd44649b4b6d1abb1d6de23c19fbe010c42"}
Jul 01 15:01:00.619473 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:00.619306    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq" event={"ID":"a136e7df-7ce1-434c-a08b-a08a8fc6bba4","Type":"ContainerStarted","Data":"634876fad9a83eea58797ee8d4fc5de146936258e2ce106b8027ccc27be15eb3"}
Jul 01 15:01:00.619473 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:00.619460    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq"
Jul 01 15:01:00.619596 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:00.619519    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq"
Jul 01 15:01:00.619596 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:00.619552    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq" event={"ID":"a136e7df-7ce1-434c-a08b-a08a8fc6bba4","Type":"ContainerStarted","Data":"39976bf7df710749af0f7708adaca7cd588e33b97ffa47c285bdaa1a3524db36"}
Jul 01 15:01:00.620772 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:00.620744    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq" podUID="a136e7df-7ce1-434c-a08b-a08a8fc6bba4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Jul 01 15:01:00.621150 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:00.621131    2551 generic.go:356] "Generic (PLEG): container finished" podID="b323f816-6b6d-4d84-bfe0-d3f00f776296" containerID="f049b2e41d4ab4fb1b45300ea1dab0b02f81d43612795915d8621cbe1535a5bd" exitCode=2
Jul 01 15:01:00.621218 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:00.621192    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q" event={"ID":"b323f816-6b6d-4d84-bfe0-d3f00f776296","Type":"ContainerDied","Data":"f049b2e41d4ab4fb1b45300ea1dab0b02f81d43612795915d8621cbe1535a5bd"}
Jul 01 15:01:00.637574 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:00.637520    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq" podStartSLOduration=1.637504942 podStartE2EDuration="1.637504942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 15:00:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 15:01:00.63593243 +0000 UTC m=+1297.964504749" watchObservedRunningTime="2026-07-01 15:01:00.637504942 +0000 UTC m=+1297.966077248"
Jul 01 15:01:01.626267 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:01.626223    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq" podUID="a136e7df-7ce1-434c-a08b-a08a8fc6bba4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Jul 01 15:01:02.186810 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:02.186767    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q" podUID="b323f816-6b6d-4d84-bfe0-d3f00f776296" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.31:8643/healthz\": dial tcp 10.133.0.31:8643: connect: connection refused"
Jul 01 15:01:02.191255 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:02.191216    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q" podUID="b323f816-6b6d-4d84-bfe0-d3f00f776296" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Jul 01 15:01:03.329295 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:03.329265    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q"
Jul 01 15:01:03.393101 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:03.393009    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b323f816-6b6d-4d84-bfe0-d3f00f776296-kube-rbac-proxy-sar-config\") pod \"b323f816-6b6d-4d84-bfe0-d3f00f776296\" (UID: \"b323f816-6b6d-4d84-bfe0-d3f00f776296\") "
Jul 01 15:01:03.393101 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:03.393054    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4gqrx\" (UniqueName: \"kubernetes.io/projected/b323f816-6b6d-4d84-bfe0-d3f00f776296-kube-api-access-4gqrx\") pod \"b323f816-6b6d-4d84-bfe0-d3f00f776296\" (UID: \"b323f816-6b6d-4d84-bfe0-d3f00f776296\") "
Jul 01 15:01:03.393101 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:03.393099    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b323f816-6b6d-4d84-bfe0-d3f00f776296-proxy-tls\") pod \"b323f816-6b6d-4d84-bfe0-d3f00f776296\" (UID: \"b323f816-6b6d-4d84-bfe0-d3f00f776296\") "
Jul 01 15:01:03.393687 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:03.393420    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b323f816-6b6d-4d84-bfe0-d3f00f776296-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "b323f816-6b6d-4d84-bfe0-d3f00f776296" (UID: "b323f816-6b6d-4d84-bfe0-d3f00f776296"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 15:01:03.395264 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:03.395243    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b323f816-6b6d-4d84-bfe0-d3f00f776296-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b323f816-6b6d-4d84-bfe0-d3f00f776296" (UID: "b323f816-6b6d-4d84-bfe0-d3f00f776296"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 15:01:03.395375 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:03.395306    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b323f816-6b6d-4d84-bfe0-d3f00f776296-kube-api-access-4gqrx" (OuterVolumeSpecName: "kube-api-access-4gqrx") pod "b323f816-6b6d-4d84-bfe0-d3f00f776296" (UID: "b323f816-6b6d-4d84-bfe0-d3f00f776296"). InnerVolumeSpecName "kube-api-access-4gqrx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 15:01:03.494569 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:03.494535    2551 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b323f816-6b6d-4d84-bfe0-d3f00f776296-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 15:01:03.494569 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:03.494563    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gqrx\" (UniqueName: \"kubernetes.io/projected/b323f816-6b6d-4d84-bfe0-d3f00f776296-kube-api-access-4gqrx\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 15:01:03.494569 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:03.494573    2551 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b323f816-6b6d-4d84-bfe0-d3f00f776296-proxy-tls\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 15:01:03.635233 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:03.635197    2551 generic.go:356] "Generic (PLEG): container finished" podID="b323f816-6b6d-4d84-bfe0-d3f00f776296" containerID="5b6e4cee605d6e500a260ba98ad5f588dcd0719f267316d6974b50a467384a49" exitCode=0
Jul 01 15:01:03.635439 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:03.635249    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q"
Jul 01 15:01:03.635439 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:03.635258    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q" event={"ID":"b323f816-6b6d-4d84-bfe0-d3f00f776296","Type":"ContainerDied","Data":"5b6e4cee605d6e500a260ba98ad5f588dcd0719f267316d6974b50a467384a49"}
Jul 01 15:01:03.635439 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:03.635318    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q" event={"ID":"b323f816-6b6d-4d84-bfe0-d3f00f776296","Type":"ContainerDied","Data":"74191a13d67dbbd2b53f7cae4f997c415af11ee6ede874235e4cdf44486f30bd"}
Jul 01 15:01:03.635439 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:03.635355    2551 scope.go:117] "RemoveContainer" containerID="f049b2e41d4ab4fb1b45300ea1dab0b02f81d43612795915d8621cbe1535a5bd"
Jul 01 15:01:03.644309 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:03.644293    2551 scope.go:117] "RemoveContainer" containerID="5b6e4cee605d6e500a260ba98ad5f588dcd0719f267316d6974b50a467384a49"
Jul 01 15:01:03.651727 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:03.651707    2551 scope.go:117] "RemoveContainer" containerID="f049b2e41d4ab4fb1b45300ea1dab0b02f81d43612795915d8621cbe1535a5bd"
Jul 01 15:01:03.651999 ip-10-0-139-55 kubenswrapper[2551]: E0701 15:01:03.651982    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f049b2e41d4ab4fb1b45300ea1dab0b02f81d43612795915d8621cbe1535a5bd\": container with ID starting with f049b2e41d4ab4fb1b45300ea1dab0b02f81d43612795915d8621cbe1535a5bd not found: ID does not exist" containerID="f049b2e41d4ab4fb1b45300ea1dab0b02f81d43612795915d8621cbe1535a5bd"
Jul 01 15:01:03.652039 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:03.652007    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f049b2e41d4ab4fb1b45300ea1dab0b02f81d43612795915d8621cbe1535a5bd"} err="failed to get container status \"f049b2e41d4ab4fb1b45300ea1dab0b02f81d43612795915d8621cbe1535a5bd\": rpc error: code = NotFound desc = could not find container \"f049b2e41d4ab4fb1b45300ea1dab0b02f81d43612795915d8621cbe1535a5bd\": container with ID starting with f049b2e41d4ab4fb1b45300ea1dab0b02f81d43612795915d8621cbe1535a5bd not found: ID does not exist"
Jul 01 15:01:03.652039 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:03.652025    2551 scope.go:117] "RemoveContainer" containerID="5b6e4cee605d6e500a260ba98ad5f588dcd0719f267316d6974b50a467384a49"
Jul 01 15:01:03.652245 ip-10-0-139-55 kubenswrapper[2551]: E0701 15:01:03.652228    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b6e4cee605d6e500a260ba98ad5f588dcd0719f267316d6974b50a467384a49\": container with ID starting with 5b6e4cee605d6e500a260ba98ad5f588dcd0719f267316d6974b50a467384a49 not found: ID does not exist" containerID="5b6e4cee605d6e500a260ba98ad5f588dcd0719f267316d6974b50a467384a49"
Jul 01 15:01:03.652280 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:03.652251    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b6e4cee605d6e500a260ba98ad5f588dcd0719f267316d6974b50a467384a49"} err="failed to get container status \"5b6e4cee605d6e500a260ba98ad5f588dcd0719f267316d6974b50a467384a49\": rpc error: code = NotFound desc = could not find container \"5b6e4cee605d6e500a260ba98ad5f588dcd0719f267316d6974b50a467384a49\": container with ID starting with 5b6e4cee605d6e500a260ba98ad5f588dcd0719f267316d6974b50a467384a49 not found: ID does not exist"
Jul 01 15:01:03.657881 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:03.657853    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q"]
Jul 01 15:01:03.666588 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:03.666447    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-0d08f-predictor-7b6f897897-5bl7q"]
Jul 01 15:01:05.103485 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:05.103446    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b323f816-6b6d-4d84-bfe0-d3f00f776296" path="/var/lib/kubelet/pods/b323f816-6b6d-4d84-bfe0-d3f00f776296/volumes"
Jul 01 15:01:06.630784 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:06.630756    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq"
Jul 01 15:01:06.631215 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:06.631177    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq" podUID="a136e7df-7ce1-434c-a08b-a08a8fc6bba4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Jul 01 15:01:16.631775 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:16.631735    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq" podUID="a136e7df-7ce1-434c-a08b-a08a8fc6bba4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Jul 01 15:01:20.193799 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:20.193764    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7"]
Jul 01 15:01:20.194297 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:20.194046    2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7" podUID="be6143cc-5b12-440c-9a8b-d562b6f874bf" containerName="kserve-container" containerID="cri-o://c3c9781939becb336d133baa6cbe5ebff46624224304ab8db2e1b748fdf467e9" gracePeriod=30
Jul 01 15:01:20.194297 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:20.194151    2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7" podUID="be6143cc-5b12-440c-9a8b-d562b6f874bf" containerName="kube-rbac-proxy" containerID="cri-o://da14dbfa2450f7086763563fef893fb1f094592883f219c0bb67a6369dfb1722" gracePeriod=30
Jul 01 15:01:20.197096 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:20.197066    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn"]
Jul 01 15:01:20.197439 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:20.197424    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b323f816-6b6d-4d84-bfe0-d3f00f776296" containerName="kube-rbac-proxy"
Jul 01 15:01:20.197439 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:20.197440    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="b323f816-6b6d-4d84-bfe0-d3f00f776296" containerName="kube-rbac-proxy"
Jul 01 15:01:20.197695 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:20.197458    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b323f816-6b6d-4d84-bfe0-d3f00f776296" containerName="kserve-container"
Jul 01 15:01:20.197695 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:20.197465    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="b323f816-6b6d-4d84-bfe0-d3f00f776296" containerName="kserve-container"
Jul 01 15:01:20.197695 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:20.197520    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="b323f816-6b6d-4d84-bfe0-d3f00f776296" containerName="kube-rbac-proxy"
Jul 01 15:01:20.197695 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:20.197531    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="b323f816-6b6d-4d84-bfe0-d3f00f776296" containerName="kserve-container"
Jul 01 15:01:20.202511 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:20.202489    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn"
Jul 01 15:01:20.204963 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:20.204936    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-9eaf2-kube-rbac-proxy-sar-config\""
Jul 01 15:01:20.205068 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:20.205029    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-9eaf2-predictor-serving-cert\""
Jul 01 15:01:20.214170 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:20.214140    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn"]
Jul 01 15:01:20.340828 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:20.340773    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktxwb\" (UniqueName: \"kubernetes.io/projected/3f561b4d-7097-4caf-988c-d5176f698299-kube-api-access-ktxwb\") pod \"success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn\" (UID: \"3f561b4d-7097-4caf-988c-d5176f698299\") " pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn"
Jul 01 15:01:20.341010 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:20.340875    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f561b4d-7097-4caf-988c-d5176f698299-proxy-tls\") pod \"success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn\" (UID: \"3f561b4d-7097-4caf-988c-d5176f698299\") " pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn"
Jul 01 15:01:20.341010 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:20.340910    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3f561b4d-7097-4caf-988c-d5176f698299-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn\" (UID: \"3f561b4d-7097-4caf-988c-d5176f698299\") " pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn"
Jul 01 15:01:20.441466 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:20.441420    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktxwb\" (UniqueName: \"kubernetes.io/projected/3f561b4d-7097-4caf-988c-d5176f698299-kube-api-access-ktxwb\") pod \"success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn\" (UID: \"3f561b4d-7097-4caf-988c-d5176f698299\") " pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn"
Jul 01 15:01:20.441466 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:20.441467    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f561b4d-7097-4caf-988c-d5176f698299-proxy-tls\") pod \"success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn\" (UID: \"3f561b4d-7097-4caf-988c-d5176f698299\") " pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn"
Jul 01 15:01:20.441784 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:20.441487    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3f561b4d-7097-4caf-988c-d5176f698299-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn\" (UID: \"3f561b4d-7097-4caf-988c-d5176f698299\") " pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn"
Jul 01 15:01:20.442314 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:20.442289    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3f561b4d-7097-4caf-988c-d5176f698299-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn\" (UID: \"3f561b4d-7097-4caf-988c-d5176f698299\") " pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn"
Jul 01 15:01:20.444118 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:20.444055    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f561b4d-7097-4caf-988c-d5176f698299-proxy-tls\") pod \"success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn\" (UID: \"3f561b4d-7097-4caf-988c-d5176f698299\") " pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn"
Jul 01 15:01:20.451367 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:20.451330    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktxwb\" (UniqueName: \"kubernetes.io/projected/3f561b4d-7097-4caf-988c-d5176f698299-kube-api-access-ktxwb\") pod \"success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn\" (UID: \"3f561b4d-7097-4caf-988c-d5176f698299\") " pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn"
Jul 01 15:01:20.520919 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:20.520869    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn"
Jul 01 15:01:20.669946 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:20.668782    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn"]
Jul 01 15:01:20.710735 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:20.710702    2551 generic.go:356] "Generic (PLEG): container finished" podID="be6143cc-5b12-440c-9a8b-d562b6f874bf" containerID="da14dbfa2450f7086763563fef893fb1f094592883f219c0bb67a6369dfb1722" exitCode=2
Jul 01 15:01:20.710899 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:20.710774    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7" event={"ID":"be6143cc-5b12-440c-9a8b-d562b6f874bf","Type":"ContainerDied","Data":"da14dbfa2450f7086763563fef893fb1f094592883f219c0bb67a6369dfb1722"}
Jul 01 15:01:20.712014 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:20.711986    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn" event={"ID":"3f561b4d-7097-4caf-988c-d5176f698299","Type":"ContainerStarted","Data":"37467ec9399452cb576edaa9719490a1dca798739e6e036740e0fe17273a40e8"}
Jul 01 15:01:21.719108 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:21.719070    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn" event={"ID":"3f561b4d-7097-4caf-988c-d5176f698299","Type":"ContainerStarted","Data":"0ae24912997cc91ed36dd9122f72bc5171091c86dc49873f0bc47b07bf8e0547"}
Jul 01 15:01:21.719530 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:21.719142    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn" event={"ID":"3f561b4d-7097-4caf-988c-d5176f698299","Type":"ContainerStarted","Data":"520bb58d13cade13f6d21b6dd1628e7d2fd3097e448be5630952fd6c052a4d73"}
Jul 01 15:01:21.719530 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:21.719177    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn"
Jul 01 15:01:21.719530 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:21.719338    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn"
Jul 01 15:01:21.720787 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:21.720762    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn" podUID="3f561b4d-7097-4caf-988c-d5176f698299" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jul 01 15:01:21.738140 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:21.738080    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn" podStartSLOduration=1.738061962 podStartE2EDuration="1.738061962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 15:01:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 15:01:21.736917764 +0000 UTC m=+1319.065490075" watchObservedRunningTime="2026-07-01 15:01:21.738061962 +0000 UTC m=+1319.066634335"
Jul 01 15:01:22.371564 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:22.371516    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7" podUID="be6143cc-5b12-440c-9a8b-d562b6f874bf" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.33:8643/healthz\": dial tcp 10.133.0.33:8643: connect: connection refused"
Jul 01 15:01:22.724118 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:22.724080    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn" podUID="3f561b4d-7097-4caf-988c-d5176f698299" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jul 01 15:01:23.732014 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:23.731981    2551 generic.go:356] "Generic (PLEG): container finished" podID="be6143cc-5b12-440c-9a8b-d562b6f874bf" containerID="c3c9781939becb336d133baa6cbe5ebff46624224304ab8db2e1b748fdf467e9" exitCode=0
Jul 01 15:01:23.732389 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:23.732052    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7" event={"ID":"be6143cc-5b12-440c-9a8b-d562b6f874bf","Type":"ContainerDied","Data":"c3c9781939becb336d133baa6cbe5ebff46624224304ab8db2e1b748fdf467e9"}
Jul 01 15:01:23.851498 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:23.851475    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7"
Jul 01 15:01:23.976094 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:23.976063    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be6143cc-5b12-440c-9a8b-d562b6f874bf-proxy-tls\") pod \"be6143cc-5b12-440c-9a8b-d562b6f874bf\" (UID: \"be6143cc-5b12-440c-9a8b-d562b6f874bf\") "
Jul 01 15:01:23.976253 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:23.976113    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbggq\" (UniqueName: \"kubernetes.io/projected/be6143cc-5b12-440c-9a8b-d562b6f874bf-kube-api-access-xbggq\") pod \"be6143cc-5b12-440c-9a8b-d562b6f874bf\" (UID: \"be6143cc-5b12-440c-9a8b-d562b6f874bf\") "
Jul 01 15:01:23.976253 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:23.976159    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/be6143cc-5b12-440c-9a8b-d562b6f874bf-kube-rbac-proxy-sar-config\") pod \"be6143cc-5b12-440c-9a8b-d562b6f874bf\" (UID: \"be6143cc-5b12-440c-9a8b-d562b6f874bf\") "
Jul 01 15:01:23.976554 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:23.976530    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be6143cc-5b12-440c-9a8b-d562b6f874bf-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "be6143cc-5b12-440c-9a8b-d562b6f874bf" (UID: "be6143cc-5b12-440c-9a8b-d562b6f874bf"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 15:01:23.978259 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:23.978237    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be6143cc-5b12-440c-9a8b-d562b6f874bf-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "be6143cc-5b12-440c-9a8b-d562b6f874bf" (UID: "be6143cc-5b12-440c-9a8b-d562b6f874bf"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 15:01:23.978351 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:23.978327    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be6143cc-5b12-440c-9a8b-d562b6f874bf-kube-api-access-xbggq" (OuterVolumeSpecName: "kube-api-access-xbggq") pod "be6143cc-5b12-440c-9a8b-d562b6f874bf" (UID: "be6143cc-5b12-440c-9a8b-d562b6f874bf"). InnerVolumeSpecName "kube-api-access-xbggq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 15:01:24.076862 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:24.076816    2551 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be6143cc-5b12-440c-9a8b-d562b6f874bf-proxy-tls\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 15:01:24.076862 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:24.076851    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbggq\" (UniqueName: \"kubernetes.io/projected/be6143cc-5b12-440c-9a8b-d562b6f874bf-kube-api-access-xbggq\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 15:01:24.076862 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:24.076860    2551 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/be6143cc-5b12-440c-9a8b-d562b6f874bf-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 15:01:24.738515 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:24.738483    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7"
Jul 01 15:01:24.738515 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:24.738496    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7" event={"ID":"be6143cc-5b12-440c-9a8b-d562b6f874bf","Type":"ContainerDied","Data":"2d93469799a72ed73f0b884654cc3ecb78d18523ca7dd5d8016b597a6ba90ba4"}
Jul 01 15:01:24.739025 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:24.738606    2551 scope.go:117] "RemoveContainer" containerID="da14dbfa2450f7086763563fef893fb1f094592883f219c0bb67a6369dfb1722"
Jul 01 15:01:24.748075 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:24.748056    2551 scope.go:117] "RemoveContainer" containerID="c3c9781939becb336d133baa6cbe5ebff46624224304ab8db2e1b748fdf467e9"
Jul 01 15:01:24.760527 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:24.760498    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7"]
Jul 01 15:01:24.765277 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:24.765252    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-7bec4-predictor-7d98f4b667-58sb7"]
Jul 01 15:01:25.103104 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:25.103023    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be6143cc-5b12-440c-9a8b-d562b6f874bf" path="/var/lib/kubelet/pods/be6143cc-5b12-440c-9a8b-d562b6f874bf/volumes"
Jul 01 15:01:26.631182 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:26.631145    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq" podUID="a136e7df-7ce1-434c-a08b-a08a8fc6bba4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Jul 01 15:01:27.728303 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:27.728273    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn"
Jul 01 15:01:27.728756 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:27.728730    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn" podUID="3f561b4d-7097-4caf-988c-d5176f698299" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jul 01 15:01:36.631779 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:36.631737    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq" podUID="a136e7df-7ce1-434c-a08b-a08a8fc6bba4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Jul 01 15:01:37.728791 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:37.728742    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn" podUID="3f561b4d-7097-4caf-988c-d5176f698299" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jul 01 15:01:46.631607 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:46.631516    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq" podUID="a136e7df-7ce1-434c-a08b-a08a8fc6bba4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Jul 01 15:01:47.729273 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:47.729233    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn" podUID="3f561b4d-7097-4caf-988c-d5176f698299" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jul 01 15:01:56.632126 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:56.632092    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq"
Jul 01 15:01:57.728843 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:01:57.728804    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn" podUID="3f561b4d-7097-4caf-988c-d5176f698299" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jul 01 15:02:07.730292 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:07.730263    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn"
Jul 01 15:02:20.038355 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.038317    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq"]
Jul 01 15:02:20.038852 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.038635    2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq" podUID="a136e7df-7ce1-434c-a08b-a08a8fc6bba4" containerName="kserve-container" containerID="cri-o://634876fad9a83eea58797ee8d4fc5de146936258e2ce106b8027ccc27be15eb3" gracePeriod=30
Jul 01 15:02:20.039078 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.039045    2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq" podUID="a136e7df-7ce1-434c-a08b-a08a8fc6bba4" containerName="kube-rbac-proxy" containerID="cri-o://c6651d39789a2096dcaf426a0aa8fbd44649b4b6d1abb1d6de23c19fbe010c42" gracePeriod=30
Jul 01 15:02:20.067784 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.067744    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk"]
Jul 01 15:02:20.068101 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.068083    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be6143cc-5b12-440c-9a8b-d562b6f874bf" containerName="kube-rbac-proxy"
Jul 01 15:02:20.068101 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.068100    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="be6143cc-5b12-440c-9a8b-d562b6f874bf" containerName="kube-rbac-proxy"
Jul 01 15:02:20.068256 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.068126    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be6143cc-5b12-440c-9a8b-d562b6f874bf" containerName="kserve-container"
Jul 01 15:02:20.068256 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.068132    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="be6143cc-5b12-440c-9a8b-d562b6f874bf" containerName="kserve-container"
Jul 01 15:02:20.068256 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.068190    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="be6143cc-5b12-440c-9a8b-d562b6f874bf" containerName="kserve-container"
Jul 01 15:02:20.068256 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.068200    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="be6143cc-5b12-440c-9a8b-d562b6f874bf" containerName="kube-rbac-proxy"
Jul 01 15:02:20.071581 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.071559    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk"
Jul 01 15:02:20.073875 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.073808    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-6e167-kube-rbac-proxy-sar-config\""
Jul 01 15:02:20.074051 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.074028    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-6e167-predictor-serving-cert\""
Jul 01 15:02:20.081433 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.081408    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk"]
Jul 01 15:02:20.160606 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.160574    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e184d628-16a0-4d10-a4d4-eecfeb8b19a4-proxy-tls\") pod \"success-200-isvc-6e167-predictor-765674cfdc-r74gk\" (UID: \"e184d628-16a0-4d10-a4d4-eecfeb8b19a4\") " pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk"
Jul 01 15:02:20.160781 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.160701    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxxwv\" (UniqueName: \"kubernetes.io/projected/e184d628-16a0-4d10-a4d4-eecfeb8b19a4-kube-api-access-gxxwv\") pod \"success-200-isvc-6e167-predictor-765674cfdc-r74gk\" (UID: \"e184d628-16a0-4d10-a4d4-eecfeb8b19a4\") " pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk"
Jul 01 15:02:20.160781 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.160731    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e184d628-16a0-4d10-a4d4-eecfeb8b19a4-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-6e167-predictor-765674cfdc-r74gk\" (UID: \"e184d628-16a0-4d10-a4d4-eecfeb8b19a4\") " pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk"
Jul 01 15:02:20.261553 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.261518    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e184d628-16a0-4d10-a4d4-eecfeb8b19a4-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-6e167-predictor-765674cfdc-r74gk\" (UID: \"e184d628-16a0-4d10-a4d4-eecfeb8b19a4\") " pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk"
Jul 01 15:02:20.261758 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.261609    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e184d628-16a0-4d10-a4d4-eecfeb8b19a4-proxy-tls\") pod \"success-200-isvc-6e167-predictor-765674cfdc-r74gk\" (UID: \"e184d628-16a0-4d10-a4d4-eecfeb8b19a4\") " pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk"
Jul 01 15:02:20.261758 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.261690    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxxwv\" (UniqueName: \"kubernetes.io/projected/e184d628-16a0-4d10-a4d4-eecfeb8b19a4-kube-api-access-gxxwv\") pod \"success-200-isvc-6e167-predictor-765674cfdc-r74gk\" (UID: \"e184d628-16a0-4d10-a4d4-eecfeb8b19a4\") " pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk"
Jul 01 15:02:20.262247 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.262217    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e184d628-16a0-4d10-a4d4-eecfeb8b19a4-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-6e167-predictor-765674cfdc-r74gk\" (UID: \"e184d628-16a0-4d10-a4d4-eecfeb8b19a4\") " pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk"
Jul 01 15:02:20.264276 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.264248    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e184d628-16a0-4d10-a4d4-eecfeb8b19a4-proxy-tls\") pod \"success-200-isvc-6e167-predictor-765674cfdc-r74gk\" (UID: \"e184d628-16a0-4d10-a4d4-eecfeb8b19a4\") " pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk"
Jul 01 15:02:20.275911 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.275883    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxxwv\" (UniqueName: \"kubernetes.io/projected/e184d628-16a0-4d10-a4d4-eecfeb8b19a4-kube-api-access-gxxwv\") pod \"success-200-isvc-6e167-predictor-765674cfdc-r74gk\" (UID: \"e184d628-16a0-4d10-a4d4-eecfeb8b19a4\") " pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk"
Jul 01 15:02:20.384323 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.384235    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk"
Jul 01 15:02:20.532961 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.532931    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk"]
Jul 01 15:02:20.535124 ip-10-0-139-55 kubenswrapper[2551]: W0701 15:02:20.535093    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode184d628_16a0_4d10_a4d4_eecfeb8b19a4.slice/crio-4796fa18f8082c7784c05132833bde7e0396c47d0cfb76808f24ca39c7504539 WatchSource:0}: Error finding container 4796fa18f8082c7784c05132833bde7e0396c47d0cfb76808f24ca39c7504539: Status 404 returned error can't find the container with id 4796fa18f8082c7784c05132833bde7e0396c47d0cfb76808f24ca39c7504539
Jul 01 15:02:20.996766 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.996731    2551 generic.go:356] "Generic (PLEG): container finished" podID="a136e7df-7ce1-434c-a08b-a08a8fc6bba4" containerID="c6651d39789a2096dcaf426a0aa8fbd44649b4b6d1abb1d6de23c19fbe010c42" exitCode=2
Jul 01 15:02:20.997013 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.996814    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq" event={"ID":"a136e7df-7ce1-434c-a08b-a08a8fc6bba4","Type":"ContainerDied","Data":"c6651d39789a2096dcaf426a0aa8fbd44649b4b6d1abb1d6de23c19fbe010c42"}
Jul 01 15:02:20.998817 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.998788    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk" event={"ID":"e184d628-16a0-4d10-a4d4-eecfeb8b19a4","Type":"ContainerStarted","Data":"c6f8ddd4933ed9c8bc2e731986152a87610126dc68e41962042a105ecbf5b147"}
Jul 01 15:02:20.998971 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.998849    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk" event={"ID":"e184d628-16a0-4d10-a4d4-eecfeb8b19a4","Type":"ContainerStarted","Data":"f63a3f6202f8e9cdeadfae3a9d4b841092589292dedf0615025ca51634d86150"}
Jul 01 15:02:20.998971 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.998883    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk" event={"ID":"e184d628-16a0-4d10-a4d4-eecfeb8b19a4","Type":"ContainerStarted","Data":"4796fa18f8082c7784c05132833bde7e0396c47d0cfb76808f24ca39c7504539"}
Jul 01 15:02:20.998971 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.998915    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk"
Jul 01 15:02:20.999110 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:20.999038    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk"
Jul 01 15:02:21.000424 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:21.000392    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk" podUID="e184d628-16a0-4d10-a4d4-eecfeb8b19a4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Jul 01 15:02:21.017438 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:21.017374    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk" podStartSLOduration=1.017355241 podStartE2EDuration="1.017355241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 15:02:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 15:02:21.01696089 +0000 UTC m=+1378.345533196" watchObservedRunningTime="2026-07-01 15:02:21.017355241 +0000 UTC m=+1378.345927547"
Jul 01 15:02:21.626696 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:21.626653    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq" podUID="a136e7df-7ce1-434c-a08b-a08a8fc6bba4" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.34:8643/healthz\": dial tcp 10.133.0.34:8643: connect: connection refused"
Jul 01 15:02:22.003560 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:22.003522    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk" podUID="e184d628-16a0-4d10-a4d4-eecfeb8b19a4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Jul 01 15:02:23.602530 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:23.602498    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq"
Jul 01 15:02:23.791792 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:23.791754    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frg6r\" (UniqueName: \"kubernetes.io/projected/a136e7df-7ce1-434c-a08b-a08a8fc6bba4-kube-api-access-frg6r\") pod \"a136e7df-7ce1-434c-a08b-a08a8fc6bba4\" (UID: \"a136e7df-7ce1-434c-a08b-a08a8fc6bba4\") "
Jul 01 15:02:23.791999 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:23.791828    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a136e7df-7ce1-434c-a08b-a08a8fc6bba4-kube-rbac-proxy-sar-config\") pod \"a136e7df-7ce1-434c-a08b-a08a8fc6bba4\" (UID: \"a136e7df-7ce1-434c-a08b-a08a8fc6bba4\") "
Jul 01 15:02:23.791999 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:23.791945    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a136e7df-7ce1-434c-a08b-a08a8fc6bba4-proxy-tls\") pod \"a136e7df-7ce1-434c-a08b-a08a8fc6bba4\" (UID: \"a136e7df-7ce1-434c-a08b-a08a8fc6bba4\") "
Jul 01 15:02:23.792184 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:23.792153    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a136e7df-7ce1-434c-a08b-a08a8fc6bba4-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "a136e7df-7ce1-434c-a08b-a08a8fc6bba4" (UID: "a136e7df-7ce1-434c-a08b-a08a8fc6bba4"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 15:02:23.793947 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:23.793914    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a136e7df-7ce1-434c-a08b-a08a8fc6bba4-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "a136e7df-7ce1-434c-a08b-a08a8fc6bba4" (UID: "a136e7df-7ce1-434c-a08b-a08a8fc6bba4"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 15:02:23.794074 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:23.793967    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a136e7df-7ce1-434c-a08b-a08a8fc6bba4-kube-api-access-frg6r" (OuterVolumeSpecName: "kube-api-access-frg6r") pod "a136e7df-7ce1-434c-a08b-a08a8fc6bba4" (UID: "a136e7df-7ce1-434c-a08b-a08a8fc6bba4"). InnerVolumeSpecName "kube-api-access-frg6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 15:02:23.892919 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:23.892874    2551 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a136e7df-7ce1-434c-a08b-a08a8fc6bba4-proxy-tls\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 15:02:23.892919 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:23.892913    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frg6r\" (UniqueName: \"kubernetes.io/projected/a136e7df-7ce1-434c-a08b-a08a8fc6bba4-kube-api-access-frg6r\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 15:02:23.892919 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:23.892927    2551 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a136e7df-7ce1-434c-a08b-a08a8fc6bba4-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 15:02:24.013990 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:24.013956    2551 generic.go:356] "Generic (PLEG): container finished" podID="a136e7df-7ce1-434c-a08b-a08a8fc6bba4" containerID="634876fad9a83eea58797ee8d4fc5de146936258e2ce106b8027ccc27be15eb3" exitCode=0
Jul 01 15:02:24.014161 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:24.014014    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq"
Jul 01 15:02:24.014161 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:24.014045    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq" event={"ID":"a136e7df-7ce1-434c-a08b-a08a8fc6bba4","Type":"ContainerDied","Data":"634876fad9a83eea58797ee8d4fc5de146936258e2ce106b8027ccc27be15eb3"}
Jul 01 15:02:24.014161 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:24.014115    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq" event={"ID":"a136e7df-7ce1-434c-a08b-a08a8fc6bba4","Type":"ContainerDied","Data":"39976bf7df710749af0f7708adaca7cd588e33b97ffa47c285bdaa1a3524db36"}
Jul 01 15:02:24.014161 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:24.014146    2551 scope.go:117] "RemoveContainer" containerID="c6651d39789a2096dcaf426a0aa8fbd44649b4b6d1abb1d6de23c19fbe010c42"
Jul 01 15:02:24.023029 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:24.023001    2551 scope.go:117] "RemoveContainer" containerID="634876fad9a83eea58797ee8d4fc5de146936258e2ce106b8027ccc27be15eb3"
Jul 01 15:02:24.031006 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:24.030984    2551 scope.go:117] "RemoveContainer" containerID="c6651d39789a2096dcaf426a0aa8fbd44649b4b6d1abb1d6de23c19fbe010c42"
Jul 01 15:02:24.031300 ip-10-0-139-55 kubenswrapper[2551]: E0701 15:02:24.031282    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6651d39789a2096dcaf426a0aa8fbd44649b4b6d1abb1d6de23c19fbe010c42\": container with ID starting with c6651d39789a2096dcaf426a0aa8fbd44649b4b6d1abb1d6de23c19fbe010c42 not found: ID does not exist" containerID="c6651d39789a2096dcaf426a0aa8fbd44649b4b6d1abb1d6de23c19fbe010c42"
Jul 01 15:02:24.031399 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:24.031311    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6651d39789a2096dcaf426a0aa8fbd44649b4b6d1abb1d6de23c19fbe010c42"} err="failed to get container status \"c6651d39789a2096dcaf426a0aa8fbd44649b4b6d1abb1d6de23c19fbe010c42\": rpc error: code = NotFound desc = could not find container \"c6651d39789a2096dcaf426a0aa8fbd44649b4b6d1abb1d6de23c19fbe010c42\": container with ID starting with c6651d39789a2096dcaf426a0aa8fbd44649b4b6d1abb1d6de23c19fbe010c42 not found: ID does not exist"
Jul 01 15:02:24.031399 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:24.031331    2551 scope.go:117] "RemoveContainer" containerID="634876fad9a83eea58797ee8d4fc5de146936258e2ce106b8027ccc27be15eb3"
Jul 01 15:02:24.031580 ip-10-0-139-55 kubenswrapper[2551]: E0701 15:02:24.031563    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"634876fad9a83eea58797ee8d4fc5de146936258e2ce106b8027ccc27be15eb3\": container with ID starting with 634876fad9a83eea58797ee8d4fc5de146936258e2ce106b8027ccc27be15eb3 not found: ID does not exist" containerID="634876fad9a83eea58797ee8d4fc5de146936258e2ce106b8027ccc27be15eb3"
Jul 01 15:02:24.031651 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:24.031586    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"634876fad9a83eea58797ee8d4fc5de146936258e2ce106b8027ccc27be15eb3"} err="failed to get container status \"634876fad9a83eea58797ee8d4fc5de146936258e2ce106b8027ccc27be15eb3\": rpc error: code = NotFound desc = could not find container \"634876fad9a83eea58797ee8d4fc5de146936258e2ce106b8027ccc27be15eb3\": container with ID starting with 634876fad9a83eea58797ee8d4fc5de146936258e2ce106b8027ccc27be15eb3 not found: ID does not exist"
Jul 01 15:02:24.036994 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:24.036967    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq"]
Jul 01 15:02:24.039880 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:24.039856    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-102b8-predictor-784fbddbc5-gqzqq"]
Jul 01 15:02:25.103629 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:25.103584    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a136e7df-7ce1-434c-a08b-a08a8fc6bba4" path="/var/lib/kubelet/pods/a136e7df-7ce1-434c-a08b-a08a8fc6bba4/volumes"
Jul 01 15:02:27.007660 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:27.007615    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk"
Jul 01 15:02:27.008185 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:27.008158    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk" podUID="e184d628-16a0-4d10-a4d4-eecfeb8b19a4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Jul 01 15:02:37.008713 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:37.008659    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk" podUID="e184d628-16a0-4d10-a4d4-eecfeb8b19a4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Jul 01 15:02:47.008941 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:47.008897    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk" podUID="e184d628-16a0-4d10-a4d4-eecfeb8b19a4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Jul 01 15:02:57.009015 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:02:57.008974    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk" podUID="e184d628-16a0-4d10-a4d4-eecfeb8b19a4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Jul 01 15:03:07.008484 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:03:07.008439    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk" podUID="e184d628-16a0-4d10-a4d4-eecfeb8b19a4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Jul 01 15:03:17.008851 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:03:17.008766    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk"
Jul 01 15:04:23.151663 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:04:23.151614    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/ovn-acl-logging/0.log"
Jul 01 15:04:23.155894 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:04:23.155873    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/ovn-acl-logging/0.log"
Jul 01 15:09:23.176042 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:09:23.176006    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/ovn-acl-logging/0.log"
Jul 01 15:09:23.181169 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:09:23.181144    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/ovn-acl-logging/0.log"
Jul 01 15:10:35.064304 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:35.064262    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn"]
Jul 01 15:10:35.066830 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:35.064515    2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn" podUID="3f561b4d-7097-4caf-988c-d5176f698299" containerName="kserve-container" containerID="cri-o://520bb58d13cade13f6d21b6dd1628e7d2fd3097e448be5630952fd6c052a4d73" gracePeriod=30
Jul 01 15:10:35.066830 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:35.064557    2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn" podUID="3f561b4d-7097-4caf-988c-d5176f698299" containerName="kube-rbac-proxy" containerID="cri-o://0ae24912997cc91ed36dd9122f72bc5171091c86dc49873f0bc47b07bf8e0547" gracePeriod=30
Jul 01 15:10:35.118038 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:35.118005    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk"]
Jul 01 15:10:35.118344 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:35.118332    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a136e7df-7ce1-434c-a08b-a08a8fc6bba4" containerName="kube-rbac-proxy"
Jul 01 15:10:35.118409 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:35.118345    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="a136e7df-7ce1-434c-a08b-a08a8fc6bba4" containerName="kube-rbac-proxy"
Jul 01 15:10:35.118409 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:35.118359    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a136e7df-7ce1-434c-a08b-a08a8fc6bba4" containerName="kserve-container"
Jul 01 15:10:35.118409 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:35.118395    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="a136e7df-7ce1-434c-a08b-a08a8fc6bba4" containerName="kserve-container"
Jul 01 15:10:35.118512 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:35.118467    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="a136e7df-7ce1-434c-a08b-a08a8fc6bba4" containerName="kube-rbac-proxy"
Jul 01 15:10:35.118512 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:35.118475    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="a136e7df-7ce1-434c-a08b-a08a8fc6bba4" containerName="kserve-container"
Jul 01 15:10:35.121880 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:35.121852    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk"
Jul 01 15:10:35.124276 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:35.124243    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-6f42a-predictor-serving-cert\""
Jul 01 15:10:35.124445 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:35.124291    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-6f42a-kube-rbac-proxy-sar-config\""
Jul 01 15:10:35.131996 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:35.131957    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk"]
Jul 01 15:10:35.196803 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:35.196747    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aab1fdfa-2add-4de3-8545-7f304858df06-proxy-tls\") pod \"success-200-isvc-6f42a-predictor-bc5969967-ft9tk\" (UID: \"aab1fdfa-2add-4de3-8545-7f304858df06\") " pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk"
Jul 01 15:10:35.196987 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:35.196844    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwnm2\" (UniqueName: \"kubernetes.io/projected/aab1fdfa-2add-4de3-8545-7f304858df06-kube-api-access-pwnm2\") pod \"success-200-isvc-6f42a-predictor-bc5969967-ft9tk\" (UID: \"aab1fdfa-2add-4de3-8545-7f304858df06\") " pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk"
Jul 01 15:10:35.196987 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:35.196907    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aab1fdfa-2add-4de3-8545-7f304858df06-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-6f42a-predictor-bc5969967-ft9tk\" (UID: \"aab1fdfa-2add-4de3-8545-7f304858df06\") " pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk"
Jul 01 15:10:35.297359 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:35.297317    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aab1fdfa-2add-4de3-8545-7f304858df06-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-6f42a-predictor-bc5969967-ft9tk\" (UID: \"aab1fdfa-2add-4de3-8545-7f304858df06\") " pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk"
Jul 01 15:10:35.297573 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:35.297389    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aab1fdfa-2add-4de3-8545-7f304858df06-proxy-tls\") pod \"success-200-isvc-6f42a-predictor-bc5969967-ft9tk\" (UID: \"aab1fdfa-2add-4de3-8545-7f304858df06\") " pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk"
Jul 01 15:10:35.297573 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:35.297434    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwnm2\" (UniqueName: \"kubernetes.io/projected/aab1fdfa-2add-4de3-8545-7f304858df06-kube-api-access-pwnm2\") pod \"success-200-isvc-6f42a-predictor-bc5969967-ft9tk\" (UID: \"aab1fdfa-2add-4de3-8545-7f304858df06\") " pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk"
Jul 01 15:10:35.297573 ip-10-0-139-55 kubenswrapper[2551]: E0701 15:10:35.297562    2551 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-serving-cert: secret "success-200-isvc-6f42a-predictor-serving-cert" not found
Jul 01 15:10:35.297781 ip-10-0-139-55 kubenswrapper[2551]: E0701 15:10:35.297664    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aab1fdfa-2add-4de3-8545-7f304858df06-proxy-tls podName:aab1fdfa-2add-4de3-8545-7f304858df06 nodeName:}" failed. No retries permitted until 2026-07-01 15:10:35.797644703 +0000 UTC m=+1873.126216988 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/aab1fdfa-2add-4de3-8545-7f304858df06-proxy-tls") pod "success-200-isvc-6f42a-predictor-bc5969967-ft9tk" (UID: "aab1fdfa-2add-4de3-8545-7f304858df06") : secret "success-200-isvc-6f42a-predictor-serving-cert" not found
Jul 01 15:10:35.298051 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:35.298030    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aab1fdfa-2add-4de3-8545-7f304858df06-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-6f42a-predictor-bc5969967-ft9tk\" (UID: \"aab1fdfa-2add-4de3-8545-7f304858df06\") " pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk"
Jul 01 15:10:35.306661 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:35.306610    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwnm2\" (UniqueName: \"kubernetes.io/projected/aab1fdfa-2add-4de3-8545-7f304858df06-kube-api-access-pwnm2\") pod \"success-200-isvc-6f42a-predictor-bc5969967-ft9tk\" (UID: \"aab1fdfa-2add-4de3-8545-7f304858df06\") " pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk"
Jul 01 15:10:35.802942 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:35.802903    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aab1fdfa-2add-4de3-8545-7f304858df06-proxy-tls\") pod \"success-200-isvc-6f42a-predictor-bc5969967-ft9tk\" (UID: \"aab1fdfa-2add-4de3-8545-7f304858df06\") " pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk"
Jul 01 15:10:35.805500 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:35.805468    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aab1fdfa-2add-4de3-8545-7f304858df06-proxy-tls\") pod \"success-200-isvc-6f42a-predictor-bc5969967-ft9tk\" (UID: \"aab1fdfa-2add-4de3-8545-7f304858df06\") " pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk"
Jul 01 15:10:36.034115 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:36.034074    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk"
Jul 01 15:10:36.172085 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:36.172058    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk"]
Jul 01 15:10:36.174407 ip-10-0-139-55 kubenswrapper[2551]: W0701 15:10:36.174379    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaab1fdfa_2add_4de3_8545_7f304858df06.slice/crio-a246d5e8dd75e8adb28a342f66f27fb0321773cd4154215a13102c8f570b8d4f WatchSource:0}: Error finding container a246d5e8dd75e8adb28a342f66f27fb0321773cd4154215a13102c8f570b8d4f: Status 404 returned error can't find the container with id a246d5e8dd75e8adb28a342f66f27fb0321773cd4154215a13102c8f570b8d4f
Jul 01 15:10:36.181782 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:36.181755    2551 generic.go:356] "Generic (PLEG): container finished" podID="3f561b4d-7097-4caf-988c-d5176f698299" containerID="0ae24912997cc91ed36dd9122f72bc5171091c86dc49873f0bc47b07bf8e0547" exitCode=2
Jul 01 15:10:36.181851 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:36.181820    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn" event={"ID":"3f561b4d-7097-4caf-988c-d5176f698299","Type":"ContainerDied","Data":"0ae24912997cc91ed36dd9122f72bc5171091c86dc49873f0bc47b07bf8e0547"}
Jul 01 15:10:36.183051 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:36.183029    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk" event={"ID":"aab1fdfa-2add-4de3-8545-7f304858df06","Type":"ContainerStarted","Data":"a246d5e8dd75e8adb28a342f66f27fb0321773cd4154215a13102c8f570b8d4f"}
Jul 01 15:10:37.195167 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:37.195125    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk" event={"ID":"aab1fdfa-2add-4de3-8545-7f304858df06","Type":"ContainerStarted","Data":"cabff28973131d1f5e5828e2871e0011513e4adf40dfda509addffcf848f93ac"}
Jul 01 15:10:37.195666 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:37.195228    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk"
Jul 01 15:10:37.195666 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:37.195352    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk"
Jul 01 15:10:37.195666 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:37.195382    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk" event={"ID":"aab1fdfa-2add-4de3-8545-7f304858df06","Type":"ContainerStarted","Data":"8eabda05d72ee1e46e2cca6241c62048e5ef961eaf0046bbb027758b555e0823"}
Jul 01 15:10:37.196904 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:37.196875    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk" podUID="aab1fdfa-2add-4de3-8545-7f304858df06" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Jul 01 15:10:37.218495 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:37.218432    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk" podStartSLOduration=2.218412854 podStartE2EDuration="2.218412854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 15:10:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 15:10:37.217012021 +0000 UTC m=+1874.545584329" watchObservedRunningTime="2026-07-01 15:10:37.218412854 +0000 UTC m=+1874.546985160"
Jul 01 15:10:37.724683 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:37.724610    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn" podUID="3f561b4d-7097-4caf-988c-d5176f698299" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.35:8643/healthz\": dial tcp 10.133.0.35:8643: connect: connection refused"
Jul 01 15:10:37.728887 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:37.728860    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn" podUID="3f561b4d-7097-4caf-988c-d5176f698299" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jul 01 15:10:38.200105 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:38.200015    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk" podUID="aab1fdfa-2add-4de3-8545-7f304858df06" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Jul 01 15:10:38.726968 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:38.726943    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn"
Jul 01 15:10:38.830480 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:38.830375    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ktxwb\" (UniqueName: \"kubernetes.io/projected/3f561b4d-7097-4caf-988c-d5176f698299-kube-api-access-ktxwb\") pod \"3f561b4d-7097-4caf-988c-d5176f698299\" (UID: \"3f561b4d-7097-4caf-988c-d5176f698299\") "
Jul 01 15:10:38.830480 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:38.830463    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f561b4d-7097-4caf-988c-d5176f698299-proxy-tls\") pod \"3f561b4d-7097-4caf-988c-d5176f698299\" (UID: \"3f561b4d-7097-4caf-988c-d5176f698299\") "
Jul 01 15:10:38.830711 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:38.830488    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3f561b4d-7097-4caf-988c-d5176f698299-kube-rbac-proxy-sar-config\") pod \"3f561b4d-7097-4caf-988c-d5176f698299\" (UID: \"3f561b4d-7097-4caf-988c-d5176f698299\") "
Jul 01 15:10:38.830916 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:38.830884    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f561b4d-7097-4caf-988c-d5176f698299-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "3f561b4d-7097-4caf-988c-d5176f698299" (UID: "3f561b4d-7097-4caf-988c-d5176f698299"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 15:10:38.832597 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:38.832567    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f561b4d-7097-4caf-988c-d5176f698299-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "3f561b4d-7097-4caf-988c-d5176f698299" (UID: "3f561b4d-7097-4caf-988c-d5176f698299"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 15:10:38.832702 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:38.832642    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f561b4d-7097-4caf-988c-d5176f698299-kube-api-access-ktxwb" (OuterVolumeSpecName: "kube-api-access-ktxwb") pod "3f561b4d-7097-4caf-988c-d5176f698299" (UID: "3f561b4d-7097-4caf-988c-d5176f698299"). InnerVolumeSpecName "kube-api-access-ktxwb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 15:10:38.931231 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:38.931191    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktxwb\" (UniqueName: \"kubernetes.io/projected/3f561b4d-7097-4caf-988c-d5176f698299-kube-api-access-ktxwb\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 15:10:38.931231 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:38.931225    2551 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f561b4d-7097-4caf-988c-d5176f698299-proxy-tls\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 15:10:38.931231 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:38.931236    2551 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3f561b4d-7097-4caf-988c-d5176f698299-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 15:10:39.205927 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:39.205889    2551 generic.go:356] "Generic (PLEG): container finished" podID="3f561b4d-7097-4caf-988c-d5176f698299" containerID="520bb58d13cade13f6d21b6dd1628e7d2fd3097e448be5630952fd6c052a4d73" exitCode=0
Jul 01 15:10:39.206369 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:39.205941    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn" event={"ID":"3f561b4d-7097-4caf-988c-d5176f698299","Type":"ContainerDied","Data":"520bb58d13cade13f6d21b6dd1628e7d2fd3097e448be5630952fd6c052a4d73"}
Jul 01 15:10:39.206369 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:39.205947    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn"
Jul 01 15:10:39.206369 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:39.206009    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn" event={"ID":"3f561b4d-7097-4caf-988c-d5176f698299","Type":"ContainerDied","Data":"37467ec9399452cb576edaa9719490a1dca798739e6e036740e0fe17273a40e8"}
Jul 01 15:10:39.206369 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:39.206036    2551 scope.go:117] "RemoveContainer" containerID="0ae24912997cc91ed36dd9122f72bc5171091c86dc49873f0bc47b07bf8e0547"
Jul 01 15:10:39.214320 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:39.214300    2551 scope.go:117] "RemoveContainer" containerID="520bb58d13cade13f6d21b6dd1628e7d2fd3097e448be5630952fd6c052a4d73"
Jul 01 15:10:39.221882 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:39.221859    2551 scope.go:117] "RemoveContainer" containerID="0ae24912997cc91ed36dd9122f72bc5171091c86dc49873f0bc47b07bf8e0547"
Jul 01 15:10:39.222163 ip-10-0-139-55 kubenswrapper[2551]: E0701 15:10:39.222142    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ae24912997cc91ed36dd9122f72bc5171091c86dc49873f0bc47b07bf8e0547\": container with ID starting with 0ae24912997cc91ed36dd9122f72bc5171091c86dc49873f0bc47b07bf8e0547 not found: ID does not exist" containerID="0ae24912997cc91ed36dd9122f72bc5171091c86dc49873f0bc47b07bf8e0547"
Jul 01 15:10:39.222213 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:39.222170    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ae24912997cc91ed36dd9122f72bc5171091c86dc49873f0bc47b07bf8e0547"} err="failed to get container status \"0ae24912997cc91ed36dd9122f72bc5171091c86dc49873f0bc47b07bf8e0547\": rpc error: code = NotFound desc = could not find container \"0ae24912997cc91ed36dd9122f72bc5171091c86dc49873f0bc47b07bf8e0547\": container with ID starting with 0ae24912997cc91ed36dd9122f72bc5171091c86dc49873f0bc47b07bf8e0547 not found: ID does not exist"
Jul 01 15:10:39.222213 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:39.222188    2551 scope.go:117] "RemoveContainer" containerID="520bb58d13cade13f6d21b6dd1628e7d2fd3097e448be5630952fd6c052a4d73"
Jul 01 15:10:39.222423 ip-10-0-139-55 kubenswrapper[2551]: E0701 15:10:39.222405    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"520bb58d13cade13f6d21b6dd1628e7d2fd3097e448be5630952fd6c052a4d73\": container with ID starting with 520bb58d13cade13f6d21b6dd1628e7d2fd3097e448be5630952fd6c052a4d73 not found: ID does not exist" containerID="520bb58d13cade13f6d21b6dd1628e7d2fd3097e448be5630952fd6c052a4d73"
Jul 01 15:10:39.222477 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:39.222432    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"520bb58d13cade13f6d21b6dd1628e7d2fd3097e448be5630952fd6c052a4d73"} err="failed to get container status \"520bb58d13cade13f6d21b6dd1628e7d2fd3097e448be5630952fd6c052a4d73\": rpc error: code = NotFound desc = could not find container \"520bb58d13cade13f6d21b6dd1628e7d2fd3097e448be5630952fd6c052a4d73\": container with ID starting with 520bb58d13cade13f6d21b6dd1628e7d2fd3097e448be5630952fd6c052a4d73 not found: ID does not exist"
Jul 01 15:10:39.227638 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:39.227596    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn"]
Jul 01 15:10:39.229763 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:39.229733    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-9eaf2-predictor-65cbb969f-rqxxn"]
Jul 01 15:10:41.103969 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:41.103933    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f561b4d-7097-4caf-988c-d5176f698299" path="/var/lib/kubelet/pods/3f561b4d-7097-4caf-988c-d5176f698299/volumes"
Jul 01 15:10:43.204652 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:43.204607    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk"
Jul 01 15:10:43.205282 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:43.205256    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk" podUID="aab1fdfa-2add-4de3-8545-7f304858df06" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Jul 01 15:10:53.205673 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:10:53.205608    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk" podUID="aab1fdfa-2add-4de3-8545-7f304858df06" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Jul 01 15:11:03.205948 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:03.205900    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk" podUID="aab1fdfa-2add-4de3-8545-7f304858df06" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Jul 01 15:11:13.205237 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:13.205193    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk" podUID="aab1fdfa-2add-4de3-8545-7f304858df06" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Jul 01 15:11:23.206505 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:23.206467    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk"
Jul 01 15:11:45.056119 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.056084    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk"]
Jul 01 15:11:45.056604 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.056312    2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk" podUID="e184d628-16a0-4d10-a4d4-eecfeb8b19a4" containerName="kserve-container" containerID="cri-o://f63a3f6202f8e9cdeadfae3a9d4b841092589292dedf0615025ca51634d86150" gracePeriod=30
Jul 01 15:11:45.056604 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.056366    2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk" podUID="e184d628-16a0-4d10-a4d4-eecfeb8b19a4" containerName="kube-rbac-proxy" containerID="cri-o://c6f8ddd4933ed9c8bc2e731986152a87610126dc68e41962042a105ecbf5b147" gracePeriod=30
Jul 01 15:11:45.060509 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.060475    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq"]
Jul 01 15:11:45.061105 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.061082    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f561b4d-7097-4caf-988c-d5176f698299" containerName="kserve-container"
Jul 01 15:11:45.061105 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.061107    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f561b4d-7097-4caf-988c-d5176f698299" containerName="kserve-container"
Jul 01 15:11:45.061259 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.061132    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f561b4d-7097-4caf-988c-d5176f698299" containerName="kube-rbac-proxy"
Jul 01 15:11:45.061259 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.061141    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f561b4d-7097-4caf-988c-d5176f698299" containerName="kube-rbac-proxy"
Jul 01 15:11:45.061259 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.061215    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f561b4d-7097-4caf-988c-d5176f698299" containerName="kserve-container"
Jul 01 15:11:45.061259 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.061226    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f561b4d-7097-4caf-988c-d5176f698299" containerName="kube-rbac-proxy"
Jul 01 15:11:45.063865 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.063848    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq"
Jul 01 15:11:45.066177 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.066154    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-f5851-predictor-serving-cert\""
Jul 01 15:11:45.066347 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.066326    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-f5851-kube-rbac-proxy-sar-config\""
Jul 01 15:11:45.086797 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.086759    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq"]
Jul 01 15:11:45.212355 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.212312    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2v9k\" (UniqueName: \"kubernetes.io/projected/3a289d83-2171-4993-86a9-d38df8649fa0-kube-api-access-d2v9k\") pod \"success-200-isvc-f5851-predictor-856647c799-rjkbq\" (UID: \"3a289d83-2171-4993-86a9-d38df8649fa0\") " pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq"
Jul 01 15:11:45.212553 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.212485    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3a289d83-2171-4993-86a9-d38df8649fa0-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-f5851-predictor-856647c799-rjkbq\" (UID: \"3a289d83-2171-4993-86a9-d38df8649fa0\") " pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq"
Jul 01 15:11:45.212642 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.212565    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3a289d83-2171-4993-86a9-d38df8649fa0-proxy-tls\") pod \"success-200-isvc-f5851-predictor-856647c799-rjkbq\" (UID: \"3a289d83-2171-4993-86a9-d38df8649fa0\") " pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq"
Jul 01 15:11:45.313643 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.313579    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3a289d83-2171-4993-86a9-d38df8649fa0-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-f5851-predictor-856647c799-rjkbq\" (UID: \"3a289d83-2171-4993-86a9-d38df8649fa0\") " pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq"
Jul 01 15:11:45.313832 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.313697    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3a289d83-2171-4993-86a9-d38df8649fa0-proxy-tls\") pod \"success-200-isvc-f5851-predictor-856647c799-rjkbq\" (UID: \"3a289d83-2171-4993-86a9-d38df8649fa0\") " pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq"
Jul 01 15:11:45.313832 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.313762    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2v9k\" (UniqueName: \"kubernetes.io/projected/3a289d83-2171-4993-86a9-d38df8649fa0-kube-api-access-d2v9k\") pod \"success-200-isvc-f5851-predictor-856647c799-rjkbq\" (UID: \"3a289d83-2171-4993-86a9-d38df8649fa0\") " pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq"
Jul 01 15:11:45.313936 ip-10-0-139-55 kubenswrapper[2551]: E0701 15:11:45.313864    2551 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-f5851-predictor-serving-cert: secret "success-200-isvc-f5851-predictor-serving-cert" not found
Jul 01 15:11:45.313986 ip-10-0-139-55 kubenswrapper[2551]: E0701 15:11:45.313939    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a289d83-2171-4993-86a9-d38df8649fa0-proxy-tls podName:3a289d83-2171-4993-86a9-d38df8649fa0 nodeName:}" failed. No retries permitted until 2026-07-01 15:11:45.813921164 +0000 UTC m=+1943.142493453 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/3a289d83-2171-4993-86a9-d38df8649fa0-proxy-tls") pod "success-200-isvc-f5851-predictor-856647c799-rjkbq" (UID: "3a289d83-2171-4993-86a9-d38df8649fa0") : secret "success-200-isvc-f5851-predictor-serving-cert" not found
Jul 01 15:11:45.314333 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.314314    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3a289d83-2171-4993-86a9-d38df8649fa0-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-f5851-predictor-856647c799-rjkbq\" (UID: \"3a289d83-2171-4993-86a9-d38df8649fa0\") " pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq"
Jul 01 15:11:45.323554 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.323516    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2v9k\" (UniqueName: \"kubernetes.io/projected/3a289d83-2171-4993-86a9-d38df8649fa0-kube-api-access-d2v9k\") pod \"success-200-isvc-f5851-predictor-856647c799-rjkbq\" (UID: \"3a289d83-2171-4993-86a9-d38df8649fa0\") " pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq"
Jul 01 15:11:45.399488 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.399452    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk"]
Jul 01 15:11:45.399771 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.399725    2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk" podUID="aab1fdfa-2add-4de3-8545-7f304858df06" containerName="kserve-container" containerID="cri-o://8eabda05d72ee1e46e2cca6241c62048e5ef961eaf0046bbb027758b555e0823" gracePeriod=30
Jul 01 15:11:45.399862 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.399756    2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk" podUID="aab1fdfa-2add-4de3-8545-7f304858df06" containerName="kube-rbac-proxy" containerID="cri-o://cabff28973131d1f5e5828e2871e0011513e4adf40dfda509addffcf848f93ac" gracePeriod=30
Jul 01 15:11:45.490814 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.490775    2551 generic.go:356] "Generic (PLEG): container finished" podID="e184d628-16a0-4d10-a4d4-eecfeb8b19a4" containerID="c6f8ddd4933ed9c8bc2e731986152a87610126dc68e41962042a105ecbf5b147" exitCode=2
Jul 01 15:11:45.491000 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.490853    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk" event={"ID":"e184d628-16a0-4d10-a4d4-eecfeb8b19a4","Type":"ContainerDied","Data":"c6f8ddd4933ed9c8bc2e731986152a87610126dc68e41962042a105ecbf5b147"}
Jul 01 15:11:45.586003 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.585964    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch"]
Jul 01 15:11:45.588927 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.588899    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch"
Jul 01 15:11:45.591483 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.591458    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-23f07-predictor-serving-cert\""
Jul 01 15:11:45.591671 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.591464    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-23f07-kube-rbac-proxy-sar-config\""
Jul 01 15:11:45.600147 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.600113    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch"]
Jul 01 15:11:45.718140 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.718094    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8j7qp\" (UniqueName: \"kubernetes.io/projected/29cf3e83-f0cf-45bc-ba30-fab178b651e4-kube-api-access-8j7qp\") pod \"success-200-isvc-23f07-predictor-56689dfcb5-clmch\" (UID: \"29cf3e83-f0cf-45bc-ba30-fab178b651e4\") " pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch"
Jul 01 15:11:45.718337 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.718149    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/29cf3e83-f0cf-45bc-ba30-fab178b651e4-proxy-tls\") pod \"success-200-isvc-23f07-predictor-56689dfcb5-clmch\" (UID: \"29cf3e83-f0cf-45bc-ba30-fab178b651e4\") " pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch"
Jul 01 15:11:45.718337 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.718209    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/29cf3e83-f0cf-45bc-ba30-fab178b651e4-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-23f07-predictor-56689dfcb5-clmch\" (UID: \"29cf3e83-f0cf-45bc-ba30-fab178b651e4\") " pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch"
Jul 01 15:11:45.818751 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.818656    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/29cf3e83-f0cf-45bc-ba30-fab178b651e4-proxy-tls\") pod \"success-200-isvc-23f07-predictor-56689dfcb5-clmch\" (UID: \"29cf3e83-f0cf-45bc-ba30-fab178b651e4\") " pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch"
Jul 01 15:11:45.818751 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.818710    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/29cf3e83-f0cf-45bc-ba30-fab178b651e4-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-23f07-predictor-56689dfcb5-clmch\" (UID: \"29cf3e83-f0cf-45bc-ba30-fab178b651e4\") " pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch"
Jul 01 15:11:45.818966 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.818755    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3a289d83-2171-4993-86a9-d38df8649fa0-proxy-tls\") pod \"success-200-isvc-f5851-predictor-856647c799-rjkbq\" (UID: \"3a289d83-2171-4993-86a9-d38df8649fa0\") " pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq"
Jul 01 15:11:45.818966 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.818780    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8j7qp\" (UniqueName: \"kubernetes.io/projected/29cf3e83-f0cf-45bc-ba30-fab178b651e4-kube-api-access-8j7qp\") pod \"success-200-isvc-23f07-predictor-56689dfcb5-clmch\" (UID: \"29cf3e83-f0cf-45bc-ba30-fab178b651e4\") " pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch"
Jul 01 15:11:45.818966 ip-10-0-139-55 kubenswrapper[2551]: E0701 15:11:45.818820    2551 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-23f07-predictor-serving-cert: secret "success-200-isvc-23f07-predictor-serving-cert" not found
Jul 01 15:11:45.818966 ip-10-0-139-55 kubenswrapper[2551]: E0701 15:11:45.818901    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29cf3e83-f0cf-45bc-ba30-fab178b651e4-proxy-tls podName:29cf3e83-f0cf-45bc-ba30-fab178b651e4 nodeName:}" failed. No retries permitted until 2026-07-01 15:11:46.318878293 +0000 UTC m=+1943.647450577 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/29cf3e83-f0cf-45bc-ba30-fab178b651e4-proxy-tls") pod "success-200-isvc-23f07-predictor-56689dfcb5-clmch" (UID: "29cf3e83-f0cf-45bc-ba30-fab178b651e4") : secret "success-200-isvc-23f07-predictor-serving-cert" not found
Jul 01 15:11:45.819434 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.819411    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/29cf3e83-f0cf-45bc-ba30-fab178b651e4-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-23f07-predictor-56689dfcb5-clmch\" (UID: \"29cf3e83-f0cf-45bc-ba30-fab178b651e4\") " pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch"
Jul 01 15:11:45.821299 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.821277    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3a289d83-2171-4993-86a9-d38df8649fa0-proxy-tls\") pod \"success-200-isvc-f5851-predictor-856647c799-rjkbq\" (UID: \"3a289d83-2171-4993-86a9-d38df8649fa0\") " pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq"
Jul 01 15:11:45.828061 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.828036    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8j7qp\" (UniqueName: \"kubernetes.io/projected/29cf3e83-f0cf-45bc-ba30-fab178b651e4-kube-api-access-8j7qp\") pod \"success-200-isvc-23f07-predictor-56689dfcb5-clmch\" (UID: \"29cf3e83-f0cf-45bc-ba30-fab178b651e4\") " pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch"
Jul 01 15:11:45.983372 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:45.983329    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq"
Jul 01 15:11:46.157840 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:46.157790    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq"]
Jul 01 15:11:46.161026 ip-10-0-139-55 kubenswrapper[2551]: W0701 15:11:46.160990    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a289d83_2171_4993_86a9_d38df8649fa0.slice/crio-206b7b8fa8bd75d5ef07f806cfc1a89b35cf725202df79e7f9e1e836d0277b28 WatchSource:0}: Error finding container 206b7b8fa8bd75d5ef07f806cfc1a89b35cf725202df79e7f9e1e836d0277b28: Status 404 returned error can't find the container with id 206b7b8fa8bd75d5ef07f806cfc1a89b35cf725202df79e7f9e1e836d0277b28
Jul 01 15:11:46.322672 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:46.322561    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/29cf3e83-f0cf-45bc-ba30-fab178b651e4-proxy-tls\") pod \"success-200-isvc-23f07-predictor-56689dfcb5-clmch\" (UID: \"29cf3e83-f0cf-45bc-ba30-fab178b651e4\") " pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch"
Jul 01 15:11:46.324798 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:46.324779    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/29cf3e83-f0cf-45bc-ba30-fab178b651e4-proxy-tls\") pod \"success-200-isvc-23f07-predictor-56689dfcb5-clmch\" (UID: \"29cf3e83-f0cf-45bc-ba30-fab178b651e4\") " pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch"
Jul 01 15:11:46.499318 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:46.499282    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq" event={"ID":"3a289d83-2171-4993-86a9-d38df8649fa0","Type":"ContainerStarted","Data":"dc1181a9d4db0f0aa682b3d8f781d4d5d7069716a6a62842ad263b3d0cfe2374"}
Jul 01 15:11:46.499491 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:46.499359    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq"
Jul 01 15:11:46.499491 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:46.499389    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq"
Jul 01 15:11:46.499491 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:46.499412    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq" event={"ID":"3a289d83-2171-4993-86a9-d38df8649fa0","Type":"ContainerStarted","Data":"15f6d05592facc00b26065ca8d007c5cd2981cb98cc610334c422eec004aad88"}
Jul 01 15:11:46.499491 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:46.499439    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq" event={"ID":"3a289d83-2171-4993-86a9-d38df8649fa0","Type":"ContainerStarted","Data":"206b7b8fa8bd75d5ef07f806cfc1a89b35cf725202df79e7f9e1e836d0277b28"}
Jul 01 15:11:46.500695 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:46.500663    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq" podUID="3a289d83-2171-4993-86a9-d38df8649fa0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Jul 01 15:11:46.501127 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:46.501107    2551 generic.go:356] "Generic (PLEG): container finished" podID="aab1fdfa-2add-4de3-8545-7f304858df06" containerID="cabff28973131d1f5e5828e2871e0011513e4adf40dfda509addffcf848f93ac" exitCode=2
Jul 01 15:11:46.501547 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:46.501140    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk" event={"ID":"aab1fdfa-2add-4de3-8545-7f304858df06","Type":"ContainerDied","Data":"cabff28973131d1f5e5828e2871e0011513e4adf40dfda509addffcf848f93ac"}
Jul 01 15:11:46.501547 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:46.501143    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch"
Jul 01 15:11:46.522775 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:46.522724    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq" podStartSLOduration=1.5227084039999998 podStartE2EDuration="1.522708404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 15:11:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 15:11:46.521923123 +0000 UTC m=+1943.850495430" watchObservedRunningTime="2026-07-01 15:11:46.522708404 +0000 UTC m=+1943.851280766"
Jul 01 15:11:46.638660 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:46.638611    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch"]
Jul 01 15:11:46.640126 ip-10-0-139-55 kubenswrapper[2551]: W0701 15:11:46.640100    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29cf3e83_f0cf_45bc_ba30_fab178b651e4.slice/crio-a212de72ecdbee706138a30cefdb9a364ca1d79b4ea388110e5fbd89661ff561 WatchSource:0}: Error finding container a212de72ecdbee706138a30cefdb9a364ca1d79b4ea388110e5fbd89661ff561: Status 404 returned error can't find the container with id a212de72ecdbee706138a30cefdb9a364ca1d79b4ea388110e5fbd89661ff561
Jul 01 15:11:47.004237 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:47.004195    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk" podUID="e184d628-16a0-4d10-a4d4-eecfeb8b19a4" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.36:8643/healthz\": dial tcp 10.133.0.36:8643: connect: connection refused"
Jul 01 15:11:47.008588 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:47.008558    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk" podUID="e184d628-16a0-4d10-a4d4-eecfeb8b19a4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Jul 01 15:11:47.507664 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:47.507610    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch" event={"ID":"29cf3e83-f0cf-45bc-ba30-fab178b651e4","Type":"ContainerStarted","Data":"40ed1383a180df6c35ce5ed54b558185a259b0d7b3dae8a913316a57ee23d847"}
Jul 01 15:11:47.508144 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:47.507795    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch"
Jul 01 15:11:47.508144 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:47.507846    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch" event={"ID":"29cf3e83-f0cf-45bc-ba30-fab178b651e4","Type":"ContainerStarted","Data":"b4e148389079e868f4e915d7a8197330b9f4b7ec93117c9546b887c5d0bc010c"}
Jul 01 15:11:47.508144 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:47.507881    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch"
Jul 01 15:11:47.508144 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:47.507909    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch" event={"ID":"29cf3e83-f0cf-45bc-ba30-fab178b651e4","Type":"ContainerStarted","Data":"a212de72ecdbee706138a30cefdb9a364ca1d79b4ea388110e5fbd89661ff561"}
Jul 01 15:11:47.508144 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:47.508047    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq" podUID="3a289d83-2171-4993-86a9-d38df8649fa0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Jul 01 15:11:47.509375 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:47.509349    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch" podUID="29cf3e83-f0cf-45bc-ba30-fab178b651e4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Jul 01 15:11:47.528259 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:47.528203    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch" podStartSLOduration=2.528187968 podStartE2EDuration="2.528187968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 15:11:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 15:11:47.526816569 +0000 UTC m=+1944.855388877" watchObservedRunningTime="2026-07-01 15:11:47.528187968 +0000 UTC m=+1944.856760274"
Jul 01 15:11:48.201125 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:48.201080    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk" podUID="aab1fdfa-2add-4de3-8545-7f304858df06" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.37:8643/healthz\": dial tcp 10.133.0.37:8643: connect: connection refused"
Jul 01 15:11:48.512248 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:48.512212    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch" podUID="29cf3e83-f0cf-45bc-ba30-fab178b651e4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Jul 01 15:11:48.715890 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:48.715864    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk"
Jul 01 15:11:48.844942 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:48.844909    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxxwv\" (UniqueName: \"kubernetes.io/projected/e184d628-16a0-4d10-a4d4-eecfeb8b19a4-kube-api-access-gxxwv\") pod \"e184d628-16a0-4d10-a4d4-eecfeb8b19a4\" (UID: \"e184d628-16a0-4d10-a4d4-eecfeb8b19a4\") "
Jul 01 15:11:48.845127 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:48.845000    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e184d628-16a0-4d10-a4d4-eecfeb8b19a4-kube-rbac-proxy-sar-config\") pod \"e184d628-16a0-4d10-a4d4-eecfeb8b19a4\" (UID: \"e184d628-16a0-4d10-a4d4-eecfeb8b19a4\") "
Jul 01 15:11:48.845127 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:48.845034    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e184d628-16a0-4d10-a4d4-eecfeb8b19a4-proxy-tls\") pod \"e184d628-16a0-4d10-a4d4-eecfeb8b19a4\" (UID: \"e184d628-16a0-4d10-a4d4-eecfeb8b19a4\") "
Jul 01 15:11:48.845389 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:48.845363    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e184d628-16a0-4d10-a4d4-eecfeb8b19a4-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "e184d628-16a0-4d10-a4d4-eecfeb8b19a4" (UID: "e184d628-16a0-4d10-a4d4-eecfeb8b19a4"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 15:11:48.847419 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:48.847273    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e184d628-16a0-4d10-a4d4-eecfeb8b19a4-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "e184d628-16a0-4d10-a4d4-eecfeb8b19a4" (UID: "e184d628-16a0-4d10-a4d4-eecfeb8b19a4"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 15:11:48.848228 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:48.848203    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e184d628-16a0-4d10-a4d4-eecfeb8b19a4-kube-api-access-gxxwv" (OuterVolumeSpecName: "kube-api-access-gxxwv") pod "e184d628-16a0-4d10-a4d4-eecfeb8b19a4" (UID: "e184d628-16a0-4d10-a4d4-eecfeb8b19a4"). InnerVolumeSpecName "kube-api-access-gxxwv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 15:11:48.940877 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:48.940851    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk"
Jul 01 15:11:48.945874 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:48.945847    2551 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e184d628-16a0-4d10-a4d4-eecfeb8b19a4-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 15:11:48.945874 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:48.945871    2551 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e184d628-16a0-4d10-a4d4-eecfeb8b19a4-proxy-tls\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 15:11:48.946045 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:48.945881    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxxwv\" (UniqueName: \"kubernetes.io/projected/e184d628-16a0-4d10-a4d4-eecfeb8b19a4-kube-api-access-gxxwv\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 15:11:49.046373 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.046275    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwnm2\" (UniqueName: \"kubernetes.io/projected/aab1fdfa-2add-4de3-8545-7f304858df06-kube-api-access-pwnm2\") pod \"aab1fdfa-2add-4de3-8545-7f304858df06\" (UID: \"aab1fdfa-2add-4de3-8545-7f304858df06\") "
Jul 01 15:11:49.046373 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.046339    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aab1fdfa-2add-4de3-8545-7f304858df06-kube-rbac-proxy-sar-config\") pod \"aab1fdfa-2add-4de3-8545-7f304858df06\" (UID: \"aab1fdfa-2add-4de3-8545-7f304858df06\") "
Jul 01 15:11:49.046563 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.046377    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aab1fdfa-2add-4de3-8545-7f304858df06-proxy-tls\") pod \"aab1fdfa-2add-4de3-8545-7f304858df06\" (UID: \"aab1fdfa-2add-4de3-8545-7f304858df06\") "
Jul 01 15:11:49.046751 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.046727    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aab1fdfa-2add-4de3-8545-7f304858df06-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "aab1fdfa-2add-4de3-8545-7f304858df06" (UID: "aab1fdfa-2add-4de3-8545-7f304858df06"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 15:11:49.048509 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.048488    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aab1fdfa-2add-4de3-8545-7f304858df06-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "aab1fdfa-2add-4de3-8545-7f304858df06" (UID: "aab1fdfa-2add-4de3-8545-7f304858df06"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 15:11:49.048556 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.048540    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aab1fdfa-2add-4de3-8545-7f304858df06-kube-api-access-pwnm2" (OuterVolumeSpecName: "kube-api-access-pwnm2") pod "aab1fdfa-2add-4de3-8545-7f304858df06" (UID: "aab1fdfa-2add-4de3-8545-7f304858df06"). InnerVolumeSpecName "kube-api-access-pwnm2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 15:11:49.147196 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.147159    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwnm2\" (UniqueName: \"kubernetes.io/projected/aab1fdfa-2add-4de3-8545-7f304858df06-kube-api-access-pwnm2\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 15:11:49.147196 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.147188    2551 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aab1fdfa-2add-4de3-8545-7f304858df06-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 15:11:49.147196 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.147198    2551 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aab1fdfa-2add-4de3-8545-7f304858df06-proxy-tls\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 15:11:49.521635 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.521590    2551 generic.go:356] "Generic (PLEG): container finished" podID="e184d628-16a0-4d10-a4d4-eecfeb8b19a4" containerID="f63a3f6202f8e9cdeadfae3a9d4b841092589292dedf0615025ca51634d86150" exitCode=0
Jul 01 15:11:49.522043 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.521670    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk"
Jul 01 15:11:49.522043 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.521681    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk" event={"ID":"e184d628-16a0-4d10-a4d4-eecfeb8b19a4","Type":"ContainerDied","Data":"f63a3f6202f8e9cdeadfae3a9d4b841092589292dedf0615025ca51634d86150"}
Jul 01 15:11:49.522043 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.521739    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk" event={"ID":"e184d628-16a0-4d10-a4d4-eecfeb8b19a4","Type":"ContainerDied","Data":"4796fa18f8082c7784c05132833bde7e0396c47d0cfb76808f24ca39c7504539"}
Jul 01 15:11:49.522043 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.521768    2551 scope.go:117] "RemoveContainer" containerID="c6f8ddd4933ed9c8bc2e731986152a87610126dc68e41962042a105ecbf5b147"
Jul 01 15:11:49.523446 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.523422    2551 generic.go:356] "Generic (PLEG): container finished" podID="aab1fdfa-2add-4de3-8545-7f304858df06" containerID="8eabda05d72ee1e46e2cca6241c62048e5ef961eaf0046bbb027758b555e0823" exitCode=0
Jul 01 15:11:49.523540 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.523493    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk"
Jul 01 15:11:49.523540 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.523512    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk" event={"ID":"aab1fdfa-2add-4de3-8545-7f304858df06","Type":"ContainerDied","Data":"8eabda05d72ee1e46e2cca6241c62048e5ef961eaf0046bbb027758b555e0823"}
Jul 01 15:11:49.523674 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.523600    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk" event={"ID":"aab1fdfa-2add-4de3-8545-7f304858df06","Type":"ContainerDied","Data":"a246d5e8dd75e8adb28a342f66f27fb0321773cd4154215a13102c8f570b8d4f"}
Jul 01 15:11:49.530435 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.530419    2551 scope.go:117] "RemoveContainer" containerID="f63a3f6202f8e9cdeadfae3a9d4b841092589292dedf0615025ca51634d86150"
Jul 01 15:11:49.537864 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.537822    2551 scope.go:117] "RemoveContainer" containerID="c6f8ddd4933ed9c8bc2e731986152a87610126dc68e41962042a105ecbf5b147"
Jul 01 15:11:49.538149 ip-10-0-139-55 kubenswrapper[2551]: E0701 15:11:49.538124    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6f8ddd4933ed9c8bc2e731986152a87610126dc68e41962042a105ecbf5b147\": container with ID starting with c6f8ddd4933ed9c8bc2e731986152a87610126dc68e41962042a105ecbf5b147 not found: ID does not exist" containerID="c6f8ddd4933ed9c8bc2e731986152a87610126dc68e41962042a105ecbf5b147"
Jul 01 15:11:49.538330 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.538303    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6f8ddd4933ed9c8bc2e731986152a87610126dc68e41962042a105ecbf5b147"} err="failed to get container status \"c6f8ddd4933ed9c8bc2e731986152a87610126dc68e41962042a105ecbf5b147\": rpc error: code = NotFound desc = could not find container \"c6f8ddd4933ed9c8bc2e731986152a87610126dc68e41962042a105ecbf5b147\": container with ID starting with c6f8ddd4933ed9c8bc2e731986152a87610126dc68e41962042a105ecbf5b147 not found: ID does not exist"
Jul 01 15:11:49.538330 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.538329    2551 scope.go:117] "RemoveContainer" containerID="f63a3f6202f8e9cdeadfae3a9d4b841092589292dedf0615025ca51634d86150"
Jul 01 15:11:49.538599 ip-10-0-139-55 kubenswrapper[2551]: E0701 15:11:49.538581    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f63a3f6202f8e9cdeadfae3a9d4b841092589292dedf0615025ca51634d86150\": container with ID starting with f63a3f6202f8e9cdeadfae3a9d4b841092589292dedf0615025ca51634d86150 not found: ID does not exist" containerID="f63a3f6202f8e9cdeadfae3a9d4b841092589292dedf0615025ca51634d86150"
Jul 01 15:11:49.538667 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.538606    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f63a3f6202f8e9cdeadfae3a9d4b841092589292dedf0615025ca51634d86150"} err="failed to get container status \"f63a3f6202f8e9cdeadfae3a9d4b841092589292dedf0615025ca51634d86150\": rpc error: code = NotFound desc = could not find container \"f63a3f6202f8e9cdeadfae3a9d4b841092589292dedf0615025ca51634d86150\": container with ID starting with f63a3f6202f8e9cdeadfae3a9d4b841092589292dedf0615025ca51634d86150 not found: ID does not exist"
Jul 01 15:11:49.538667 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.538642    2551 scope.go:117] "RemoveContainer" containerID="cabff28973131d1f5e5828e2871e0011513e4adf40dfda509addffcf848f93ac"
Jul 01 15:11:49.539406 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.539388    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk"]
Jul 01 15:11:49.543212 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.543195    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6e167-predictor-765674cfdc-r74gk"]
Jul 01 15:11:49.546396 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.546382    2551 scope.go:117] "RemoveContainer" containerID="8eabda05d72ee1e46e2cca6241c62048e5ef961eaf0046bbb027758b555e0823"
Jul 01 15:11:49.554179 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.554158    2551 scope.go:117] "RemoveContainer" containerID="cabff28973131d1f5e5828e2871e0011513e4adf40dfda509addffcf848f93ac"
Jul 01 15:11:49.554448 ip-10-0-139-55 kubenswrapper[2551]: E0701 15:11:49.554429    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cabff28973131d1f5e5828e2871e0011513e4adf40dfda509addffcf848f93ac\": container with ID starting with cabff28973131d1f5e5828e2871e0011513e4adf40dfda509addffcf848f93ac not found: ID does not exist" containerID="cabff28973131d1f5e5828e2871e0011513e4adf40dfda509addffcf848f93ac"
Jul 01 15:11:49.554501 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.554456    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cabff28973131d1f5e5828e2871e0011513e4adf40dfda509addffcf848f93ac"} err="failed to get container status \"cabff28973131d1f5e5828e2871e0011513e4adf40dfda509addffcf848f93ac\": rpc error: code = NotFound desc = could not find container \"cabff28973131d1f5e5828e2871e0011513e4adf40dfda509addffcf848f93ac\": container with ID starting with cabff28973131d1f5e5828e2871e0011513e4adf40dfda509addffcf848f93ac not found: ID does not exist"
Jul 01 15:11:49.554501 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.554474    2551 scope.go:117] "RemoveContainer" containerID="8eabda05d72ee1e46e2cca6241c62048e5ef961eaf0046bbb027758b555e0823"
Jul 01 15:11:49.554584 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.554518    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk"]
Jul 01 15:11:49.554685 ip-10-0-139-55 kubenswrapper[2551]: E0701 15:11:49.554670    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8eabda05d72ee1e46e2cca6241c62048e5ef961eaf0046bbb027758b555e0823\": container with ID starting with 8eabda05d72ee1e46e2cca6241c62048e5ef961eaf0046bbb027758b555e0823 not found: ID does not exist" containerID="8eabda05d72ee1e46e2cca6241c62048e5ef961eaf0046bbb027758b555e0823"
Jul 01 15:11:49.554722 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.554690    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8eabda05d72ee1e46e2cca6241c62048e5ef961eaf0046bbb027758b555e0823"} err="failed to get container status \"8eabda05d72ee1e46e2cca6241c62048e5ef961eaf0046bbb027758b555e0823\": rpc error: code = NotFound desc = could not find container \"8eabda05d72ee1e46e2cca6241c62048e5ef961eaf0046bbb027758b555e0823\": container with ID starting with 8eabda05d72ee1e46e2cca6241c62048e5ef961eaf0046bbb027758b555e0823 not found: ID does not exist"
Jul 01 15:11:49.560163 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:49.560141    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6f42a-predictor-bc5969967-ft9tk"]
Jul 01 15:11:51.103129 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:51.103094    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aab1fdfa-2add-4de3-8545-7f304858df06" path="/var/lib/kubelet/pods/aab1fdfa-2add-4de3-8545-7f304858df06/volumes"
Jul 01 15:11:51.103525 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:51.103507    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e184d628-16a0-4d10-a4d4-eecfeb8b19a4" path="/var/lib/kubelet/pods/e184d628-16a0-4d10-a4d4-eecfeb8b19a4/volumes"
Jul 01 15:11:52.512612 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:52.512572    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq"
Jul 01 15:11:52.513173 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:52.513143    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq" podUID="3a289d83-2171-4993-86a9-d38df8649fa0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Jul 01 15:11:53.516855 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:53.516827    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch"
Jul 01 15:11:53.517425 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:11:53.517395    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch" podUID="29cf3e83-f0cf-45bc-ba30-fab178b651e4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Jul 01 15:12:02.513481 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:12:02.513440    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq" podUID="3a289d83-2171-4993-86a9-d38df8649fa0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Jul 01 15:12:03.517949 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:12:03.517908    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch" podUID="29cf3e83-f0cf-45bc-ba30-fab178b651e4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Jul 01 15:12:12.513295 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:12:12.513256    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq" podUID="3a289d83-2171-4993-86a9-d38df8649fa0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Jul 01 15:12:13.518098 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:12:13.518054    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch" podUID="29cf3e83-f0cf-45bc-ba30-fab178b651e4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Jul 01 15:12:22.513189 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:12:22.513143    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq" podUID="3a289d83-2171-4993-86a9-d38df8649fa0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Jul 01 15:12:23.517335 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:12:23.517296    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch" podUID="29cf3e83-f0cf-45bc-ba30-fab178b651e4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Jul 01 15:12:32.514006 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:12:32.513959    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq" podUID="3a289d83-2171-4993-86a9-d38df8649fa0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Jul 01 15:12:33.517959 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:12:33.517919    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch" podUID="29cf3e83-f0cf-45bc-ba30-fab178b651e4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Jul 01 15:12:42.514838 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:12:42.514797    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq"
Jul 01 15:12:43.517848 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:12:43.517817    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch"
Jul 01 15:14:23.201864 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:14:23.201830    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/ovn-acl-logging/0.log"
Jul 01 15:14:23.206846 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:14:23.206820    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/ovn-acl-logging/0.log"
Jul 01 15:19:23.226410 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:19:23.226382    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/ovn-acl-logging/0.log"
Jul 01 15:19:23.232244 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:19:23.232220    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/ovn-acl-logging/0.log"
Jul 01 15:21:10.236307 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:21:10.236272    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch"]
Jul 01 15:21:10.236909 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:21:10.236511    2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch" podUID="29cf3e83-f0cf-45bc-ba30-fab178b651e4" containerName="kserve-container" containerID="cri-o://b4e148389079e868f4e915d7a8197330b9f4b7ec93117c9546b887c5d0bc010c" gracePeriod=30
Jul 01 15:21:10.236909 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:21:10.236569    2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch" podUID="29cf3e83-f0cf-45bc-ba30-fab178b651e4" containerName="kube-rbac-proxy" containerID="cri-o://40ed1383a180df6c35ce5ed54b558185a259b0d7b3dae8a913316a57ee23d847" gracePeriod=30
Jul 01 15:21:10.954155 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:21:10.954118    2551 generic.go:356] "Generic (PLEG): container finished" podID="29cf3e83-f0cf-45bc-ba30-fab178b651e4" containerID="40ed1383a180df6c35ce5ed54b558185a259b0d7b3dae8a913316a57ee23d847" exitCode=2
Jul 01 15:21:10.954339 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:21:10.954187    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch" event={"ID":"29cf3e83-f0cf-45bc-ba30-fab178b651e4","Type":"ContainerDied","Data":"40ed1383a180df6c35ce5ed54b558185a259b0d7b3dae8a913316a57ee23d847"}
Jul 01 15:21:13.488659 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:21:13.488616    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch"
Jul 01 15:21:13.558457 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:21:13.558357    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8j7qp\" (UniqueName: \"kubernetes.io/projected/29cf3e83-f0cf-45bc-ba30-fab178b651e4-kube-api-access-8j7qp\") pod \"29cf3e83-f0cf-45bc-ba30-fab178b651e4\" (UID: \"29cf3e83-f0cf-45bc-ba30-fab178b651e4\") "
Jul 01 15:21:13.558457 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:21:13.558400    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/29cf3e83-f0cf-45bc-ba30-fab178b651e4-proxy-tls\") pod \"29cf3e83-f0cf-45bc-ba30-fab178b651e4\" (UID: \"29cf3e83-f0cf-45bc-ba30-fab178b651e4\") "
Jul 01 15:21:13.558457 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:21:13.558422    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/29cf3e83-f0cf-45bc-ba30-fab178b651e4-kube-rbac-proxy-sar-config\") pod \"29cf3e83-f0cf-45bc-ba30-fab178b651e4\" (UID: \"29cf3e83-f0cf-45bc-ba30-fab178b651e4\") "
Jul 01 15:21:13.558850 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:21:13.558825    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29cf3e83-f0cf-45bc-ba30-fab178b651e4-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "29cf3e83-f0cf-45bc-ba30-fab178b651e4" (UID: "29cf3e83-f0cf-45bc-ba30-fab178b651e4"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 15:21:13.560524 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:21:13.560493    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29cf3e83-f0cf-45bc-ba30-fab178b651e4-kube-api-access-8j7qp" (OuterVolumeSpecName: "kube-api-access-8j7qp") pod "29cf3e83-f0cf-45bc-ba30-fab178b651e4" (UID: "29cf3e83-f0cf-45bc-ba30-fab178b651e4"). InnerVolumeSpecName "kube-api-access-8j7qp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 15:21:13.560693 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:21:13.560587    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29cf3e83-f0cf-45bc-ba30-fab178b651e4-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "29cf3e83-f0cf-45bc-ba30-fab178b651e4" (UID: "29cf3e83-f0cf-45bc-ba30-fab178b651e4"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 15:21:13.659065 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:21:13.659029    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8j7qp\" (UniqueName: \"kubernetes.io/projected/29cf3e83-f0cf-45bc-ba30-fab178b651e4-kube-api-access-8j7qp\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 15:21:13.659065 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:21:13.659062    2551 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/29cf3e83-f0cf-45bc-ba30-fab178b651e4-proxy-tls\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 15:21:13.659065 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:21:13.659075    2551 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/29cf3e83-f0cf-45bc-ba30-fab178b651e4-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 15:21:13.967719 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:21:13.967575    2551 generic.go:356] "Generic (PLEG): container finished" podID="29cf3e83-f0cf-45bc-ba30-fab178b651e4" containerID="b4e148389079e868f4e915d7a8197330b9f4b7ec93117c9546b887c5d0bc010c" exitCode=0
Jul 01 15:21:13.967719 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:21:13.967644    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch"
Jul 01 15:21:13.967719 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:21:13.967667    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch" event={"ID":"29cf3e83-f0cf-45bc-ba30-fab178b651e4","Type":"ContainerDied","Data":"b4e148389079e868f4e915d7a8197330b9f4b7ec93117c9546b887c5d0bc010c"}
Jul 01 15:21:13.968004 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:21:13.967744    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch" event={"ID":"29cf3e83-f0cf-45bc-ba30-fab178b651e4","Type":"ContainerDied","Data":"a212de72ecdbee706138a30cefdb9a364ca1d79b4ea388110e5fbd89661ff561"}
Jul 01 15:21:13.968004 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:21:13.967779    2551 scope.go:117] "RemoveContainer" containerID="40ed1383a180df6c35ce5ed54b558185a259b0d7b3dae8a913316a57ee23d847"
Jul 01 15:21:13.977158 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:21:13.977139    2551 scope.go:117] "RemoveContainer" containerID="b4e148389079e868f4e915d7a8197330b9f4b7ec93117c9546b887c5d0bc010c"
Jul 01 15:21:13.984750 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:21:13.984726    2551 scope.go:117] "RemoveContainer" containerID="40ed1383a180df6c35ce5ed54b558185a259b0d7b3dae8a913316a57ee23d847"
Jul 01 15:21:13.985023 ip-10-0-139-55 kubenswrapper[2551]: E0701 15:21:13.985005    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40ed1383a180df6c35ce5ed54b558185a259b0d7b3dae8a913316a57ee23d847\": container with ID starting with 40ed1383a180df6c35ce5ed54b558185a259b0d7b3dae8a913316a57ee23d847 not found: ID does not exist" containerID="40ed1383a180df6c35ce5ed54b558185a259b0d7b3dae8a913316a57ee23d847"
Jul 01 15:21:13.985078 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:21:13.985034    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40ed1383a180df6c35ce5ed54b558185a259b0d7b3dae8a913316a57ee23d847"} err="failed to get container status \"40ed1383a180df6c35ce5ed54b558185a259b0d7b3dae8a913316a57ee23d847\": rpc error: code = NotFound desc = could not find container \"40ed1383a180df6c35ce5ed54b558185a259b0d7b3dae8a913316a57ee23d847\": container with ID starting with 40ed1383a180df6c35ce5ed54b558185a259b0d7b3dae8a913316a57ee23d847 not found: ID does not exist"
Jul 01 15:21:13.985078 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:21:13.985052    2551 scope.go:117] "RemoveContainer" containerID="b4e148389079e868f4e915d7a8197330b9f4b7ec93117c9546b887c5d0bc010c"
Jul 01 15:21:13.985315 ip-10-0-139-55 kubenswrapper[2551]: E0701 15:21:13.985297    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4e148389079e868f4e915d7a8197330b9f4b7ec93117c9546b887c5d0bc010c\": container with ID starting with b4e148389079e868f4e915d7a8197330b9f4b7ec93117c9546b887c5d0bc010c not found: ID does not exist" containerID="b4e148389079e868f4e915d7a8197330b9f4b7ec93117c9546b887c5d0bc010c"
Jul 01 15:21:13.985360 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:21:13.985327    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4e148389079e868f4e915d7a8197330b9f4b7ec93117c9546b887c5d0bc010c"} err="failed to get container status \"b4e148389079e868f4e915d7a8197330b9f4b7ec93117c9546b887c5d0bc010c\": rpc error: code = NotFound desc = could not find container \"b4e148389079e868f4e915d7a8197330b9f4b7ec93117c9546b887c5d0bc010c\": container with ID starting with b4e148389079e868f4e915d7a8197330b9f4b7ec93117c9546b887c5d0bc010c not found: ID does not exist"
Jul 01 15:21:13.989514 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:21:13.989491    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch"]
Jul 01 15:21:13.993571 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:21:13.993547    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-23f07-predictor-56689dfcb5-clmch"]
Jul 01 15:21:15.103692 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:21:15.103657    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29cf3e83-f0cf-45bc-ba30-fab178b651e4" path="/var/lib/kubelet/pods/29cf3e83-f0cf-45bc-ba30-fab178b651e4/volumes"
Jul 01 15:24:23.253739 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:24:23.253707    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/ovn-acl-logging/0.log"
Jul 01 15:24:23.260141 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:24:23.260114    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/ovn-acl-logging/0.log"
Jul 01 15:29:14.766685 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:14.766582    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq"]
Jul 01 15:29:14.769080 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:14.766857    2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq" podUID="3a289d83-2171-4993-86a9-d38df8649fa0" containerName="kserve-container" containerID="cri-o://15f6d05592facc00b26065ca8d007c5cd2981cb98cc610334c422eec004aad88" gracePeriod=30
Jul 01 15:29:14.769080 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:14.766904    2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq" podUID="3a289d83-2171-4993-86a9-d38df8649fa0" containerName="kube-rbac-proxy" containerID="cri-o://dc1181a9d4db0f0aa682b3d8f781d4d5d7069716a6a62842ad263b3d0cfe2374" gracePeriod=30
Jul 01 15:29:15.033460 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:15.033359    2551 generic.go:356] "Generic (PLEG): container finished" podID="3a289d83-2171-4993-86a9-d38df8649fa0" containerID="dc1181a9d4db0f0aa682b3d8f781d4d5d7069716a6a62842ad263b3d0cfe2374" exitCode=2
Jul 01 15:29:15.033460 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:15.033440    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq" event={"ID":"3a289d83-2171-4993-86a9-d38df8649fa0","Type":"ContainerDied","Data":"dc1181a9d4db0f0aa682b3d8f781d4d5d7069716a6a62842ad263b3d0cfe2374"}
Jul 01 15:29:17.508658 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:17.508606    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq" podUID="3a289d83-2171-4993-86a9-d38df8649fa0" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.38:8643/healthz\": dial tcp 10.133.0.38:8643: connect: connection refused"
Jul 01 15:29:18.312523 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:18.312495    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq"
Jul 01 15:29:18.358523 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:18.358419    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3a289d83-2171-4993-86a9-d38df8649fa0-proxy-tls\") pod \"3a289d83-2171-4993-86a9-d38df8649fa0\" (UID: \"3a289d83-2171-4993-86a9-d38df8649fa0\") "
Jul 01 15:29:18.358523 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:18.358515    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3a289d83-2171-4993-86a9-d38df8649fa0-kube-rbac-proxy-sar-config\") pod \"3a289d83-2171-4993-86a9-d38df8649fa0\" (UID: \"3a289d83-2171-4993-86a9-d38df8649fa0\") "
Jul 01 15:29:18.358750 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:18.358571    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2v9k\" (UniqueName: \"kubernetes.io/projected/3a289d83-2171-4993-86a9-d38df8649fa0-kube-api-access-d2v9k\") pod \"3a289d83-2171-4993-86a9-d38df8649fa0\" (UID: \"3a289d83-2171-4993-86a9-d38df8649fa0\") "
Jul 01 15:29:18.358972 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:18.358929    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a289d83-2171-4993-86a9-d38df8649fa0-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "3a289d83-2171-4993-86a9-d38df8649fa0" (UID: "3a289d83-2171-4993-86a9-d38df8649fa0"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 15:29:18.360588 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:18.360556    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a289d83-2171-4993-86a9-d38df8649fa0-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "3a289d83-2171-4993-86a9-d38df8649fa0" (UID: "3a289d83-2171-4993-86a9-d38df8649fa0"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 15:29:18.360789 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:18.360769    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a289d83-2171-4993-86a9-d38df8649fa0-kube-api-access-d2v9k" (OuterVolumeSpecName: "kube-api-access-d2v9k") pod "3a289d83-2171-4993-86a9-d38df8649fa0" (UID: "3a289d83-2171-4993-86a9-d38df8649fa0"). InnerVolumeSpecName "kube-api-access-d2v9k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 15:29:18.459582 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:18.459540    2551 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3a289d83-2171-4993-86a9-d38df8649fa0-proxy-tls\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 15:29:18.459582 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:18.459573    2551 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3a289d83-2171-4993-86a9-d38df8649fa0-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 15:29:18.459582 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:18.459587    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2v9k\" (UniqueName: \"kubernetes.io/projected/3a289d83-2171-4993-86a9-d38df8649fa0-kube-api-access-d2v9k\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\""
Jul 01 15:29:19.052838 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:19.052789    2551 generic.go:356] "Generic (PLEG): container finished" podID="3a289d83-2171-4993-86a9-d38df8649fa0" containerID="15f6d05592facc00b26065ca8d007c5cd2981cb98cc610334c422eec004aad88" exitCode=0
Jul 01 15:29:19.053269 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:19.052844    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq"
Jul 01 15:29:19.053269 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:19.052839    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq" event={"ID":"3a289d83-2171-4993-86a9-d38df8649fa0","Type":"ContainerDied","Data":"15f6d05592facc00b26065ca8d007c5cd2981cb98cc610334c422eec004aad88"}
Jul 01 15:29:19.053269 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:19.052960    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq" event={"ID":"3a289d83-2171-4993-86a9-d38df8649fa0","Type":"ContainerDied","Data":"206b7b8fa8bd75d5ef07f806cfc1a89b35cf725202df79e7f9e1e836d0277b28"}
Jul 01 15:29:19.053269 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:19.052987    2551 scope.go:117] "RemoveContainer" containerID="dc1181a9d4db0f0aa682b3d8f781d4d5d7069716a6a62842ad263b3d0cfe2374"
Jul 01 15:29:19.062125 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:19.062104    2551 scope.go:117] "RemoveContainer" containerID="15f6d05592facc00b26065ca8d007c5cd2981cb98cc610334c422eec004aad88"
Jul 01 15:29:19.070103 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:19.070080    2551 scope.go:117] "RemoveContainer" containerID="dc1181a9d4db0f0aa682b3d8f781d4d5d7069716a6a62842ad263b3d0cfe2374"
Jul 01 15:29:19.070384 ip-10-0-139-55 kubenswrapper[2551]: E0701 15:29:19.070361    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc1181a9d4db0f0aa682b3d8f781d4d5d7069716a6a62842ad263b3d0cfe2374\": container with ID starting with dc1181a9d4db0f0aa682b3d8f781d4d5d7069716a6a62842ad263b3d0cfe2374 not found: ID does not exist" containerID="dc1181a9d4db0f0aa682b3d8f781d4d5d7069716a6a62842ad263b3d0cfe2374"
Jul 01 15:29:19.070463 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:19.070397    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc1181a9d4db0f0aa682b3d8f781d4d5d7069716a6a62842ad263b3d0cfe2374"} err="failed to get container status \"dc1181a9d4db0f0aa682b3d8f781d4d5d7069716a6a62842ad263b3d0cfe2374\": rpc error: code = NotFound desc = could not find container \"dc1181a9d4db0f0aa682b3d8f781d4d5d7069716a6a62842ad263b3d0cfe2374\": container with ID starting with dc1181a9d4db0f0aa682b3d8f781d4d5d7069716a6a62842ad263b3d0cfe2374 not found: ID does not exist"
Jul 01 15:29:19.070463 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:19.070420    2551 scope.go:117] "RemoveContainer" containerID="15f6d05592facc00b26065ca8d007c5cd2981cb98cc610334c422eec004aad88"
Jul 01 15:29:19.070691 ip-10-0-139-55 kubenswrapper[2551]: E0701 15:29:19.070673    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15f6d05592facc00b26065ca8d007c5cd2981cb98cc610334c422eec004aad88\": container with ID starting with 15f6d05592facc00b26065ca8d007c5cd2981cb98cc610334c422eec004aad88 not found: ID does not exist" containerID="15f6d05592facc00b26065ca8d007c5cd2981cb98cc610334c422eec004aad88"
Jul 01 15:29:19.070751 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:19.070698    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15f6d05592facc00b26065ca8d007c5cd2981cb98cc610334c422eec004aad88"} err="failed to get container status \"15f6d05592facc00b26065ca8d007c5cd2981cb98cc610334c422eec004aad88\": rpc error: code = NotFound desc = could not find container \"15f6d05592facc00b26065ca8d007c5cd2981cb98cc610334c422eec004aad88\": container with ID starting with 15f6d05592facc00b26065ca8d007c5cd2981cb98cc610334c422eec004aad88 not found: ID does not exist"
Jul 01 15:29:19.076522 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:19.076495    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq"]
Jul 01 15:29:19.079719 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:19.079695    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f5851-predictor-856647c799-rjkbq"]
Jul 01 15:29:19.103379 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:19.103335    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a289d83-2171-4993-86a9-d38df8649fa0" path="/var/lib/kubelet/pods/3a289d83-2171-4993-86a9-d38df8649fa0/volumes"
Jul 01 15:29:23.276639 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:23.276596    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/ovn-acl-logging/0.log"
Jul 01 15:29:23.283232 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:23.283211    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/ovn-acl-logging/0.log"
Jul 01 15:29:49.333581 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:49.333542    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2lqtw_99ded651-5fce-49b7-8561-d54826fd2690/global-pull-secret-syncer/0.log"
Jul 01 15:29:49.500225 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:49.500191    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qrcb9_0710008c-6eee-4fd1-aac9-81df1341171f/konnectivity-agent/0.log"
Jul 01 15:29:49.614854 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:49.614754    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-55.ec2.internal_814e0bac5e0890277234107c5ef5089c/haproxy/0.log"
Jul 01 15:29:52.966534 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:52.966499    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-svkc6_b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1/kube-state-metrics/0.log"
Jul 01 15:29:53.013676 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:53.013639    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-svkc6_b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1/kube-rbac-proxy-main/0.log"
Jul 01 15:29:53.048270 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:53.048244    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-svkc6_b996e3e7-4810-4b61-88e8-d8e7e7c7f7a1/kube-rbac-proxy-self/0.log"
Jul 01 15:29:53.206005 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:53.205968    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b2qcb_652dda43-a27b-4fbf-b04f-ca9a0267861a/node-exporter/0.log"
Jul 01 15:29:53.281743 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:53.281709    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b2qcb_652dda43-a27b-4fbf-b04f-ca9a0267861a/kube-rbac-proxy/0.log"
Jul 01 15:29:53.337297 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:53.337270    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b2qcb_652dda43-a27b-4fbf-b04f-ca9a0267861a/init-textfile/0.log"
Jul 01 15:29:54.032100 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:54.032067    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-99wxn_00f36b77-b452-442a-917e-c49f5902ccce/prometheus-operator/0.log"
Jul 01 15:29:54.056543 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:54.056494    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-99wxn_00f36b77-b452-442a-917e-c49f5902ccce/kube-rbac-proxy/0.log"
Jul 01 15:29:54.091634 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:54.091592    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7ddc8784d4-xmvdl_05d8c9d2-2f1c-4261-8d4a-4203c652007f/prometheus-operator-admission-webhook/0.log"
Jul 01 15:29:56.855941 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:56.855892    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-56b95f574b-ghq9b_3499eb6d-ae9e-4448-af49-901e8091f9f3/console/0.log"
Jul 01 15:29:56.889550 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:56.889510    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5499b6fb8f-pm2t5_3bf70ba6-5f5c-4a13-a624-3c5227c62a01/download-server/0.log"
Jul 01 15:29:57.737615 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.737577    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wp8cx/perf-node-gather-daemonset-gvqtf"]
Jul 01 15:29:57.737911 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.737899    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e184d628-16a0-4d10-a4d4-eecfeb8b19a4" containerName="kserve-container"
Jul 01 15:29:57.737963 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.737913    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="e184d628-16a0-4d10-a4d4-eecfeb8b19a4" containerName="kserve-container"
Jul 01 15:29:57.737963 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.737923    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a289d83-2171-4993-86a9-d38df8649fa0" containerName="kube-rbac-proxy"
Jul 01 15:29:57.737963 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.737928    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a289d83-2171-4993-86a9-d38df8649fa0" containerName="kube-rbac-proxy"
Jul 01 15:29:57.737963 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.737936    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aab1fdfa-2add-4de3-8545-7f304858df06" containerName="kube-rbac-proxy"
Jul 01 15:29:57.737963 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.737942    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="aab1fdfa-2add-4de3-8545-7f304858df06" containerName="kube-rbac-proxy"
Jul 01 15:29:57.737963 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.737948    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e184d628-16a0-4d10-a4d4-eecfeb8b19a4" containerName="kube-rbac-proxy"
Jul 01 15:29:57.737963 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.737954    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="e184d628-16a0-4d10-a4d4-eecfeb8b19a4" containerName="kube-rbac-proxy"
Jul 01 15:29:57.737963 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.737959    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aab1fdfa-2add-4de3-8545-7f304858df06" containerName="kserve-container"
Jul 01 15:29:57.737963 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.737964    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="aab1fdfa-2add-4de3-8545-7f304858df06" containerName="kserve-container"
Jul 01 15:29:57.738227 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.737972    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a289d83-2171-4993-86a9-d38df8649fa0" containerName="kserve-container"
Jul 01 15:29:57.738227 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.737977    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a289d83-2171-4993-86a9-d38df8649fa0" containerName="kserve-container"
Jul 01 15:29:57.738227 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.737987    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29cf3e83-f0cf-45bc-ba30-fab178b651e4" containerName="kube-rbac-proxy"
Jul 01 15:29:57.738227 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.737992    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="29cf3e83-f0cf-45bc-ba30-fab178b651e4" containerName="kube-rbac-proxy"
Jul 01 15:29:57.738227 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.738000    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29cf3e83-f0cf-45bc-ba30-fab178b651e4" containerName="kserve-container"
Jul 01 15:29:57.738227 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.738005    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="29cf3e83-f0cf-45bc-ba30-fab178b651e4" containerName="kserve-container"
Jul 01 15:29:57.738227 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.738052    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="aab1fdfa-2add-4de3-8545-7f304858df06" containerName="kserve-container"
Jul 01 15:29:57.738227 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.738062    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a289d83-2171-4993-86a9-d38df8649fa0" containerName="kserve-container"
Jul 01 15:29:57.738227 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.738068    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="29cf3e83-f0cf-45bc-ba30-fab178b651e4" containerName="kserve-container"
Jul 01 15:29:57.738227 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.738073    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="e184d628-16a0-4d10-a4d4-eecfeb8b19a4" containerName="kserve-container"
Jul 01 15:29:57.738227 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.738078    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a289d83-2171-4993-86a9-d38df8649fa0" containerName="kube-rbac-proxy"
Jul 01 15:29:57.738227 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.738083    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="29cf3e83-f0cf-45bc-ba30-fab178b651e4" containerName="kube-rbac-proxy"
Jul 01 15:29:57.738227 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.738088    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="aab1fdfa-2add-4de3-8545-7f304858df06" containerName="kube-rbac-proxy"
Jul 01 15:29:57.738227 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.738093    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="e184d628-16a0-4d10-a4d4-eecfeb8b19a4" containerName="kube-rbac-proxy"
Jul 01 15:29:57.741139 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.741106    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-gvqtf"
Jul 01 15:29:57.744029 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.744005    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wp8cx\"/\"default-dockercfg-ddj5c\""
Jul 01 15:29:57.744181 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.744065    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wp8cx\"/\"kube-root-ca.crt\""
Jul 01 15:29:57.744888 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.744871    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wp8cx\"/\"openshift-service-ca.crt\""
Jul 01 15:29:57.752991 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.752965    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wp8cx/perf-node-gather-daemonset-gvqtf"]
Jul 01 15:29:57.895278 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.895236    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhnns\" (UniqueName: \"kubernetes.io/projected/23f98156-61ae-4e2e-ac6d-ff3e33630e18-kube-api-access-mhnns\") pod \"perf-node-gather-daemonset-gvqtf\" (UID: \"23f98156-61ae-4e2e-ac6d-ff3e33630e18\") " pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-gvqtf"
Jul 01 15:29:57.895747 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.895291    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/23f98156-61ae-4e2e-ac6d-ff3e33630e18-sys\") pod \"perf-node-gather-daemonset-gvqtf\" (UID: \"23f98156-61ae-4e2e-ac6d-ff3e33630e18\") " pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-gvqtf"
Jul 01 15:29:57.895747 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.895317    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/23f98156-61ae-4e2e-ac6d-ff3e33630e18-podres\") pod \"perf-node-gather-daemonset-gvqtf\" (UID: \"23f98156-61ae-4e2e-ac6d-ff3e33630e18\") " pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-gvqtf"
Jul 01 15:29:57.895747 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.895369    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/23f98156-61ae-4e2e-ac6d-ff3e33630e18-proc\") pod \"perf-node-gather-daemonset-gvqtf\" (UID: \"23f98156-61ae-4e2e-ac6d-ff3e33630e18\") " pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-gvqtf"
Jul 01 15:29:57.895747 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.895417    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/23f98156-61ae-4e2e-ac6d-ff3e33630e18-lib-modules\") pod \"perf-node-gather-daemonset-gvqtf\" (UID: \"23f98156-61ae-4e2e-ac6d-ff3e33630e18\") " pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-gvqtf"
Jul 01 15:29:57.996567 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.996472    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/23f98156-61ae-4e2e-ac6d-ff3e33630e18-sys\") pod \"perf-node-gather-daemonset-gvqtf\" (UID: \"23f98156-61ae-4e2e-ac6d-ff3e33630e18\") " pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-gvqtf"
Jul 01 15:29:57.996567 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.996514    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/23f98156-61ae-4e2e-ac6d-ff3e33630e18-podres\") pod \"perf-node-gather-daemonset-gvqtf\" (UID: \"23f98156-61ae-4e2e-ac6d-ff3e33630e18\") " pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-gvqtf"
Jul 01 15:29:57.996567 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.996551    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/23f98156-61ae-4e2e-ac6d-ff3e33630e18-proc\") pod \"perf-node-gather-daemonset-gvqtf\" (UID: \"23f98156-61ae-4e2e-ac6d-ff3e33630e18\") " pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-gvqtf"
Jul 01 15:29:57.996837 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.996584    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/23f98156-61ae-4e2e-ac6d-ff3e33630e18-lib-modules\") pod \"perf-node-gather-daemonset-gvqtf\" (UID: \"23f98156-61ae-4e2e-ac6d-ff3e33630e18\") " pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-gvqtf"
Jul 01 15:29:57.996837 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.996602    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/23f98156-61ae-4e2e-ac6d-ff3e33630e18-sys\") pod \"perf-node-gather-daemonset-gvqtf\" (UID: \"23f98156-61ae-4e2e-ac6d-ff3e33630e18\") " pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-gvqtf"
Jul 01 15:29:57.996837 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.996611    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhnns\" (UniqueName: \"kubernetes.io/projected/23f98156-61ae-4e2e-ac6d-ff3e33630e18-kube-api-access-mhnns\") pod \"perf-node-gather-daemonset-gvqtf\" (UID: \"23f98156-61ae-4e2e-ac6d-ff3e33630e18\") " pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-gvqtf"
Jul 01 15:29:57.996837 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.996697    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/23f98156-61ae-4e2e-ac6d-ff3e33630e18-proc\") pod \"perf-node-gather-daemonset-gvqtf\" (UID: \"23f98156-61ae-4e2e-ac6d-ff3e33630e18\") " pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-gvqtf"
Jul 01 15:29:57.996837 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.996740    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/23f98156-61ae-4e2e-ac6d-ff3e33630e18-podres\") pod \"perf-node-gather-daemonset-gvqtf\" (UID: \"23f98156-61ae-4e2e-ac6d-ff3e33630e18\") " pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-gvqtf"
Jul 01 15:29:57.996837 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:57.996747    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/23f98156-61ae-4e2e-ac6d-ff3e33630e18-lib-modules\") pod \"perf-node-gather-daemonset-gvqtf\" (UID: \"23f98156-61ae-4e2e-ac6d-ff3e33630e18\") " pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-gvqtf"
Jul 01 15:29:58.006535 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:58.006498    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhnns\" (UniqueName: \"kubernetes.io/projected/23f98156-61ae-4e2e-ac6d-ff3e33630e18-kube-api-access-mhnns\") pod \"perf-node-gather-daemonset-gvqtf\" (UID: \"23f98156-61ae-4e2e-ac6d-ff3e33630e18\") " pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-gvqtf"
Jul 01 15:29:58.051228 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:58.051188    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-gvqtf"
Jul 01 15:29:58.195437 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:58.195403    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wp8cx/perf-node-gather-daemonset-gvqtf"]
Jul 01 15:29:58.198375 ip-10-0-139-55 kubenswrapper[2551]: W0701 15:29:58.198346    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod23f98156_61ae_4e2e_ac6d_ff3e33630e18.slice/crio-d9abc7d6202c59cc28a111a76171e782f6ce74b460e8a501fa2cd70f6ca00810 WatchSource:0}: Error finding container d9abc7d6202c59cc28a111a76171e782f6ce74b460e8a501fa2cd70f6ca00810: Status 404 returned error can't find the container with id d9abc7d6202c59cc28a111a76171e782f6ce74b460e8a501fa2cd70f6ca00810
Jul 01 15:29:58.223293 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:58.223264    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-gvqtf" event={"ID":"23f98156-61ae-4e2e-ac6d-ff3e33630e18","Type":"ContainerStarted","Data":"d9abc7d6202c59cc28a111a76171e782f6ce74b460e8a501fa2cd70f6ca00810"}
Jul 01 15:29:58.398889 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:58.398836    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9grqk_1e94a284-d202-4352-82fa-dd51de2f2c4e/dns/0.log"
Jul 01 15:29:58.423463 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:58.423433    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9grqk_1e94a284-d202-4352-82fa-dd51de2f2c4e/kube-rbac-proxy/0.log"
Jul 01 15:29:58.531055 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:58.530965    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rzw6z_4bc04385-5d77-4673-a019-88bb93a250dd/dns-node-resolver/0.log"
Jul 01 15:29:59.042940 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:59.042904    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5fc59ff786-9vm7x_9632f072-3da7-4b81-9d81-715b0e36890d/registry/0.log"
Jul 01 15:29:59.090995 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:59.090967    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hn47h_49ff50e4-fbc4-4623-a5c5-458c628adef3/node-ca/0.log"
Jul 01 15:29:59.228334 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:59.228301    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-gvqtf" event={"ID":"23f98156-61ae-4e2e-ac6d-ff3e33630e18","Type":"ContainerStarted","Data":"df5a56da4f2eeeadecb8afd45182c78065e04442a0b1b0329e90d7ce0d2e994b"}
Jul 01 15:29:59.228545 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:59.228418    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-gvqtf"
Jul 01 15:29:59.245428 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:29:59.245368    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-gvqtf" podStartSLOduration=2.245350486 podStartE2EDuration="2.245350486s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 15:29:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 15:29:59.243904615 +0000 UTC m=+3036.572476943" watchObservedRunningTime="2026-07-01 15:29:59.245350486 +0000 UTC m=+3036.573922789"
Jul 01 15:30:00.279775 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:00.279740    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jw949_f93bdf31-79e7-4d7f-9956-5a5a7e9e5075/serve-healthcheck-canary/0.log"
Jul 01 15:30:00.795979 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:00.795948    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-f7wph_6ebcec99-5af8-4b1c-9b74-cec8a0712906/kube-rbac-proxy/0.log"
Jul 01 15:30:00.820012 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:00.819977    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-f7wph_6ebcec99-5af8-4b1c-9b74-cec8a0712906/exporter/0.log"
Jul 01 15:30:00.842090 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:00.842065    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-f7wph_6ebcec99-5af8-4b1c-9b74-cec8a0712906/extractor/0.log"
Jul 01 15:30:03.054774 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:03.054731    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-5d77c4c564-thtll_210ae3d0-3cde-49d7-b680-1f19fac3b5a2/manager/0.log"
Jul 01 15:30:03.120087 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:03.120056    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-fd65d7d6-5gw8k_b345ac12-4ace-42ce-9631-3769287e311e/server/0.log"
Jul 01 15:30:03.217583 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:03.217539    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-84b54b7f97-47pwn_2c675763-aa41-4746-8cab-297bf7a82d9e/manager/0.log"
Jul 01 15:30:03.290410 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:03.290378    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-64568bcd49-vm2tg_aa05ded8-91de-42fb-a8e4-62fc84f7498e/seaweedfs/0.log"
Jul 01 15:30:05.242579 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:05.242538    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wp8cx/perf-node-gather-daemonset-gvqtf"
Jul 01 15:30:09.194533 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:09.194454    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nhhr7_bc612921-fe17-41c3-93ba-2e1197f6f843/kube-multus-additional-cni-plugins/0.log"
Jul 01 15:30:09.219396 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:09.219372    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nhhr7_bc612921-fe17-41c3-93ba-2e1197f6f843/egress-router-binary-copy/0.log"
Jul 01 15:30:09.243574 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:09.243548    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nhhr7_bc612921-fe17-41c3-93ba-2e1197f6f843/cni-plugins/0.log"
Jul 01 15:30:09.266724 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:09.266681    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nhhr7_bc612921-fe17-41c3-93ba-2e1197f6f843/bond-cni-plugin/0.log"
Jul 01 15:30:09.292399 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:09.292368    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nhhr7_bc612921-fe17-41c3-93ba-2e1197f6f843/routeoverride-cni/0.log"
Jul 01 15:30:09.320588 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:09.320552    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nhhr7_bc612921-fe17-41c3-93ba-2e1197f6f843/whereabouts-cni-bincopy/0.log"
Jul 01 15:30:09.346107 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:09.346079    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nhhr7_bc612921-fe17-41c3-93ba-2e1197f6f843/whereabouts-cni/0.log"
Jul 01 15:30:09.382036 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:09.382006    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-jxqmr_dc2cfa4b-185e-424a-af8b-c715dbe39c9d/kube-multus/0.log"
Jul 01 15:30:09.554715 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:09.554687    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-tprn9_005968bd-f145-4c81-be95-b3170cafef84/network-metrics-daemon/0.log"
Jul 01 15:30:09.578816 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:09.578788    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-tprn9_005968bd-f145-4c81-be95-b3170cafef84/kube-rbac-proxy/0.log"
Jul 01 15:30:10.319130 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:10.319092    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/ovn-controller/0.log"
Jul 01 15:30:10.341097 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:10.341070    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/ovn-acl-logging/1.log"
Jul 01 15:30:10.341268 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:10.341118    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/ovn-acl-logging/0.log"
Jul 01 15:30:10.360379 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:10.360357    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/kube-rbac-proxy-node/0.log"
Jul 01 15:30:10.384374 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:10.384341    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/kube-rbac-proxy-ovn-metrics/0.log"
Jul 01 15:30:10.410837 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:10.410811    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/northd/0.log"
Jul 01 15:30:10.436754 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:10.436726    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/nbdb/0.log"
Jul 01 15:30:10.462139 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:10.462110    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/sbdb/0.log"
Jul 01 15:30:10.512439 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:10.512402    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnczw_768a96c0-935c-4629-b015-1b85b0ac3199/ovnkube-controller/0.log"
Jul 01 15:30:12.115666 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:12.115611    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-km6cc_8b8d9348-6896-4fd2-b3c1-8d841219bc43/network-check-target-container/0.log"
Jul 01 15:30:13.089363 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:13.089331    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-mgl6l_3ea52ec7-825f-4d91-9215-17d64f0b497d/iptables-alerter/0.log"
Jul 01 15:30:13.738730 ip-10-0-139-55 kubenswrapper[2551]: I0701 15:30:13.738693    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-649r6_fa1455f9-1f95-47e0-9d5f-f73c917257cf/tuned/0.log"